Using a Wiki to support Agile documentation

Currently one of the projects I’m working on is for a client in the public sector. I’m part of a team that creates specialised software to support the processes of this organization and all her partners involved in child protective services. Currently the software is being used by around 1500 personnel and this number is only expected to grow. This means that although we use an Agile way of working to develop the software, we have to create documentation for the maintenance team to maintain and if necessary further develop the application. We use a BPM suite to develop this software. As you can read from previous blog items for documentation purposes I strive to use the self explanatory models of the BPM suite as much as possible but although we can cover a great deal of the documentation need with the suite we’re using, we still have to create some extra documentation in another form than the models themselves.

More specific we document:

• User stories, to justify the need for a piece of functionality and to steer the project
• Acceptance criteria, as a test base and to give more functional detail to a piece of functionality
• System test scripts, to be able to check regression and to help document more technical decisions
• Service definition, to describe the functionality needed for interfaces to more technical parts and third party interfaces.

Using User Stories for documentation is not a straight forward process. I wonder whether the set of User Stories you use on your project, sufficiently describe the functionality of the system. I also wonder whether maintenance personnel, not initially part of the team are able to navigate the user stories and find information they need easily. In my opinion just creating a backlog does not do the trick as the set of User Stories is fairly unstructured. At the start of the project these user stories describe needed functionality and form a good basis for documentation. Later on in the project the user stories also start describing changes to the earlier functionality. If you need an overview of the functionality created you, need to read through all the user stories created about the subject.

To help with this problem, I’ve created a functional structure derived from the application. The user stories created are placed in the proper place in the structure. The structure itself comprises the processes, process tasks and lower level modules that make up the application. The names of the elements in the structure follow the names that users (and thus maintenance personnel also) find in the application (ubiquitous language). The development team links every User Story to it’s place in the structure. The result is that the User Stories can easily be found back by maintenance personnel. This enables the maintenance team, but also the development team and implementation team to retrieve the information put in the user stories and acceptance criteria. As for user stories, the structure can also be used for the system tests or any other documentation you feel is needed for maintenance.

You may ask yourself what tool is needed to create this structure? Requirements management tools are capable of doing so. The problem with these tools is that they come with a price and a learning curve. The question is, is there an inexpensive tool that is easy to learn and easy to use?  

I’ve been able to create just that: Using open source MediaWiki with CategoryTree plug-in my team is able to document the software on those parts that are not covered by the BPM suite models. Creating a sprint, adding User stories, placing these user stories in the right place of the skeleton is easy and straight forward. Navigating the structure by the maintenance team and others involved is easy and follows the structure of the application itself. Adding test scripts, storing test results and documenting service descriptions in the structure is as easy.

Working with a wiki adds to the agility of the team as it is light weight, flexible and does the job well. Working with a wiki is more easily accepted by Agile team members then more elaborate requirements tooling.  

What I’ve created is a structure for documenting software in an agile manner. I’ve created a help page with templates and guidelines on how to use this wiki. Please contact me if you’re want more information or want to get the wiki pages enabling this structure. I can help you and your team to implement this way of working, giving them a way to document for maintenance purposes.

Lean requirements in BPMS, an example

Having written quite some requirements documents and now working with current generation of BPM suites I’m seeing very interesting improvement area’s in the field of requirements. These improvement areas are supported and made possible by these powerful tools...

Hanging on to old requirements habits of specifying everything up-front and requiring sign-off of specifications before implementation, is not going to make us effective using these tools. Translating paper specifications to working software is difficult for developers, let alone for customers. As the sign-off this document becomes part of the contract, a customer will try to avoid any mistakes as he has to suffer from any rework later on. Special attention is given to getting “complete and correct” requirements … on paper...

What if we had a tool that enabled us to demonstrate  to our customers what we have in mind directly. How different, if we could create models that are readable by customers and need no extra translation by developers. How great, if we (and our maintenance teams) would always have the corresponding version of the requirements available when looking at some implementation artifact? Sounds like a beautiful future...

I'll tell you, the future has arrived when using state-of-the-art BPM suites. I would like to give you an example on how you can exploit certain features of a BPM suite to obtain a lean requirements approach.

Requirements are implementation - Be Lean!

I would like to show you how to support business processes using IBM WebSphere Lombardi. Suppose we have the following business process to handle a Buyers Purchase Order:
figure: IBM WebSphere Lombardi Process Model

The question is: What requirements artifacts do we need to describe this process sufficiently for stakeholders to understand what’s going on and for the development team to understand what to build?

Let’s start at the process model: Some stakeholders like to get an overview of the process and get exited about process models like the one shown above. They can get an overview of who is doing what and when. They get an idea of decision moments in the flow and by analysing actual information gathered from ran processes they can optimize the flows if needed. However these models don’t appeal to everyone’s imagination... Isn’t it so that especially end-users are less interested in the overview but more in the details; that they can best contribute in the requirements process if they see the end-results of a run process, when they see forms with actual information to be entered in the right sequence.

Wouldn’t it be great if we could cater both types of stakeholders?

...the good thing is, we can...

The process model shown above is the actual executable process model in Lombardi. There is no need for us requirements people to describe it anywhere else or to clarify it anywhere else. The model neatly shows who does what, when, by showing swim lanes, process steps and process flow respectively. The model speaks for itself and since process steps are well named it is clear for almost anybody in the domain to get a good idea of what is going on...

To cater the other type of users who get excited if they see actual forms where to enter information, IBM Lombardi allows us to run the process in demonstration mode, showing mock-ups of the forms (to be described later) and to let end-users test the flow of the process by entering values and selecting scenarios showing the successive actions that take place.

Brings us to what we do have to describe apart from what IBM lombardi gives us by implementing the solution. Probably we want to give an overview of the objectives to achieve using this process model and the details of the process steps. On the details of the process steps, lets take the human step to Submit a Purchase Order, in normal requirements practices we are used to writing a something like a Use Case describing the interaction of the User with the System: we would describe the data entered and the forms used, we would describe validation rules and business rules and we would describe supplementary specifications.

Using Lombardi we can take a lean approach to providing this information and only describe those assets we can’t model in the tool directly. Let’s see what we can specify in the tool apart from the process model already described above: Domain objects, User Interfaces, Business and Validation rules, Coarse steps of the UseCase.

Domain objects

With Lombardi we describe the domain objects and data to be used in the system. Often the implementation glues together information available in various legacy systems and standard applications. Some of the processing probably takes place in the legacy systems and glues together the various systems. Often information crosses departments making it important to have a mutual understanding on the meaning of each domain element and attribute. Terms like Shipping date can easily be interpreted or defined differently over departments: one department may define the shipping date being the date where the order is picked and the box is sealed, another department may define the shipping date as the date where the order actually leaves the plant. Using the descriptions of these attributes and using business rules we can clearly define what is meant with a shipping date.

IBM WebSphere Lombardi DomainModel

User Interface

BPM suites allow us to create mock-up’s of the user interfaces or the actual user interfaces used in the system itself. The mock-up’s may not contain all details in the requirements phase, but because we can show this mock-up in it’s place in the process, we have a very effective way to communicate our ideas about the process and user interfaces. In the run down of the process the user can enter data and see if the flow of forms is correct.

IBM WebSphere Lombardi User Interface

Business Rules

Also the business rules we can describe directly in the BPM suite in a readable format. see for instance the decision table below. The headers above the table refer to classes and attributes in the domain model we have created or that we update on the fly. For us requirements people it is easy to create an executable model and have the user enter data in a form, for instance the values for updatedQuantityPercent and updateUnitPricePercent. The decision table can be used in various places: in the process flow to decide on the next step or it can be used to show or hide information in a form and so on...

IBM WebSphere Lombardi validation rule

Coarse Use Case steps

We sometimes only scratch the surface by providing and creating the models described above. Besides specifying the user's interaction with the system we often have to describe how to connect to legacy systems or how a certain task breaks down in lower level activities. We can break down the functionality in smaller steps to give more detail about the processing. Often these descriptions are sufficiently describing what is needed from a functional level. If more description is needed then describe each by specifying what the user wants to acheive with this step. Also by having these models in the tool we can easily demonstrate the working of the application to the user. Do we still need a detailed description of the workflow of the user interaction? Maybe yes...

But probably no, since we can model further details in a service:
IBM WebSphere Lombardi service

Using good naming we can put extra detail to communicate with users and for the implementation team to implement these interactions in the right order.

http://www.ibm.com/developerworks/websphere/library/techarticles/1101_wang/1101_wang.html

One of the issues with this way of using the actual models as a description of the functionality is that we have to be cautious not to stick too much detail to the first level of these models. Take the service... When detailing this service developer likely add more steps if necessary. If there is too little focus on structure we may end-up with unreadable and unmaintainable models.. be aware...

By doing so our requirements process creates the actual models and gives us the power to demonstrate the implications of choices directly getting feedback from users on actual implementation rather than on paper representation of the implementation.
I would say, no time to sit back... no time to hang on to old habits...

There are now new ways to serve our customer with way more output! Take control... add new habits, start working in an agile fashion use a lean requirements approach!
Start empowering your end-users!

A norm for requirements in BMPS

As a requirements engineer I’ve often wondered myself about the balance between the effort put into creating specifications versus the actual usage of it. Off course development teams need some information to start working on, but the further you get in a project, the more the team knows what is going on and the less value your specifications have. Isn’t it so that specifications devalue even more when the system goes into a maintenance phase? Use cases which are very useful to develop the system are actually not so handy for maintenance as focus shifts from a development team wanting to know how the user interacts with the systems, to a maintenance team wanting to know where a certain form or business rule is used in order to make a change or an impact analysis. Keeping the specification a valuable and up-to-date asset takes a huge effort with unknown outcome...

I suppose anyone working in software development will confirm that software documentation often suffers from one or more problems below:

• it is nonexistent or is of poor quality;
• it is outdated;
• it may be over abundant on some aspects without a definite objective;
• it may be difficult to access when the are scattered on several location and in different formats;
• it doesn’t always get the interest from the programmers that it needs;
• it is difficult to standardize, due, for example, to project specifics.

The results of this, is that maintenance teams only trust the actual realized software system, the source code and the database tables itself. In traditional software development the source code however is hard to read. In those situations documentation is needed to explain how things work together and how requirements are implemented in the system. These descriptions however suffer from the problems listed above.

The outcome of a survey held amongst several maintenance personnel won’t surprise you, it gives an idea about what documentation artifacts are most valued by maintenance personnel probably due to the problems above:




This survey also showes that software code and comments are mostly valued by maintenance teams!

Knowing what causes the problems with documentation can help us decide on how to document in the future and what to look for in tooling to support that process.

Working in the field of BPM, I’ve found that BPM Suites offer great functionality to overcome several problems with software documentation. What the BPM suites offer in this field is based around the following four features:

1. The first good thing about BPM suites is that the models created by BPM engineers in the suite are well readable by maintenance personnel. The models are directly executable so there is no software coding needed to translate the models into the code to create an executable. The fact that documentation lacks interest of programmers becomes less relevant since the models they create are often well readable.

2. The second good thing is that these models correlate with assets we are used to specify, like a domain model, user interfaces and a process model. There is no need to create a written specification of these artifacts discussions on how to document belong to the past.

3. The next good thing is that most BPM suites keep traces between implementation artifacts of different types. This makes it possible for instance to find out in which process steps a certain business rule is used.

4. The fourth good thing about BPM suites is that requirements information is stored in the system close to the implementation artifacts helping with the access and standardization of requirements.

For a client I currently work for I’ve developed a documentation norm specifying in detail what to document in text and what to document in the suite (in this case Aquima). To get a good idea on what and how to document, I’ve started from the models often available in the suites. These models I linked to the requirements artifacts of the Rational Unified Process (RUP). Linking to RUP makes it easier for external auditors to assess the quality and completeness of the available documentation.

Please contact me for more information on the norm...