Tuesday, 25 October 2011 11:24

Does Formal Requirements Documentation Have a Place in Agile?

Written by 
Rate this item
(0 votes)

The question above is one I have faced many times, and is usually one of the first questions Business Analysts ask when confronted with an agile method. Many times this comes not so much as a question, but as a statement like, “This agile method must incorporate our documentation.” Whether this sentiment comes as a question or a statement, my answer tends to be the same: It can fit, if it has to fit.

To be certain, Formal Requirements Documents do not have a place in all, or even most, agile environments. However, just because an organization has a non-negotiable obligation to produce Formal Requirements Documents doesn’t mean agile delivery methods are not an option for that team. While Formal Requirements Documents, like Business Requirements Specifications (BRS) or Business Requirements Documents (BRD), seem to be the opposite of agile, they are often a required part of doing business. Oftentimes this is in regulated environments, such as companies that have certain documentation as part of an ISO or CMMI certification that has been achieved, or consulting firms with contractual obligations for documentation.

The path to integrating Formal Requirements Documents into agile methods includes:

  • Comprehending the underlying request or regulation for Formal Requirements Documents
  • Performing a gap-analysis of desired agile work products and required formal work products
  • Leveraging templates and automation tools to ease the publication of Formal Requirements Documents

The above steps provide a good framework for integrating agile and formal documentation. In reality, the first step is finding out if Formal Requirements Documentation really HAS to be produced. Although many demands for Formal Requirements Documentation are valid, many others are simply the knee-jerk response of a community accustomed to a lot of formal documentation. This is often one of the more difficult tasks in the process, and requires a touch of process analysis, method knowledge and a knack for organizational change management. However, once the requirement for Formal Requirements Documentation has been validated, I assure the team our new process will result in the required documentation and then immediately take the focus away from document-centric thinking. As with most agile engagements, the focus should be on requirements methods that capture what is needed to build a shared understanding; including the use of iteratively developed user stories and inclusive modelling techniques that enable active stakeholder participation.

Throughout the process of understanding the regulation or contract that is dictating the Formal Requirements Documentation, one will typically find the regulations rarely dictate how or when to capture the requirements, the level of traceability detail required or mandate formal reviews. Given these facts, there is obviously still plenty of room for agile methods within a formal framework. It is also important to get an understanding of which desired agile requirements components will comprise the Formal Requirements Documentation, and determine if there is a gap between the desired agile requirement artefacts and the regulatory constraints. Often additional artefacts will be needed, such as business rules or requirements pertaining to sensitive data. However, comprehension of the regulation is particularly critical in this area, as these additions are typically dependent on the risk or profile of the project. Typical examples might be User Stories (or Light Weight Use Cases), high-level business process sketches, and User Interface prototypes — all of which will be created in an agile and highly collaborative manner with the team and business stakeholders. Additionally, there might be mandated security rules or requirements pertaining to sensitive personal data that are typically created by an analyst based on industry specifications or standards.

Once a team has decided what its method will be, and completed a gap-analysis to confirm all the documentation needs are covered by the new method and work products, the next step is to look at how the final document will come together. This planning step is essential prior to the project kickoff so the Business Analysts and Delivery Teams will be able to focus on the design and build of the project rather than worry about the formation of documentation. The creation of lightweight templates and traceability matrices will go a long way toward ensuring a clean and compliant document can be produced from the agile work products. Many thought leaders in the space strongly advocate for templates, like Mike Cohn’s widely used User Story Template: “As a [end user role], I want [the desire] so that [the rationale].” Beyond the traditional benefits these templates give a team, they also become very important in the final assembly and publication of formal documentation.

A vital component to creating Formal Requirements Documentation from an agile project is automation for requirements traceability, version control and document publishing. There are many tools on the market that can assist with this, or custom applications or scripts can be used. Without some sort of automation, creating the Formal Requirements Documentation will become an onerous manual process. The proper level of automation will allow a team to quickly extract, assemble and publish a formal document from a collection of agile and mandated work products. The ease and quickness of automation is what allows the formal document publication to keep pace with the agile method — allowing for publication of new formal documents as often as the team sees fit or the regulations mandate. A typical scenario is to produce a formal document at the beginning and end of every sprint in order to capture the normal changes to requirements and prioritization that takes place during an agile sprint.

In summary, Formal Requirements Documentation can absolutely be integrated into the agile process, becoming just another by-product of an agile delivery team.

One good technique to achieve this is to:

  • Gain deep understanding of the regulations
  • Perform a gap-analysis of the agile work products and the mandated documents
  • Use templates and automation for traceability matrices and document publishing

While this type of hybrid method is often frowned on by some ‘agile purists,’ I find it extremely agile to acknowledge that one process does not fit all, and a bit of flexibility is usually the answer.

Don't forget to leave your comments below.


Kurt Solarte is a Managing Consultant working for IBM Australia in Sydney; focusing on Agile Development methods. Kurt recently spent eight years with IBM Global Business Services in the U.Ss as a Sr. Business Analyst; where he specialized in keeping business analysis alive and relevant in the agile delivery of eCommerce, web portal, and business analytics projects.

Read 3267 times Last modified on Monday, 02 April 2012 16:17

Comments  

 
0 # Yonico 2011-10-25 17:11
Hi Kurt great article. My company has be struggling to find the balance of what documentation is required. They are stuck with a waterfall approach and the documentation that it entails. Totally agree with the article but not sure how to reach others in the company who stuck in waterfall approach
Reply | Reply with quote | Quote
 
 
0 # Kurt McKeown 2011-10-25 17:45
Hi Kurt A very interesting and hot topic in my current project. Picture a a very large, (overly)complex organization that is used mainly to waterfall (and a little bit of RUP) projects, where processes, guidelines and templates - and CMMI - rule over efficient project execution. We now have a large project kicking-off, which shall serve as kind of a pilot for a agile project (SCRUM). I am currently trying to define exactly what you describe in your article: The way from a document centric approach to an agile one in the area of requirements engineering, and I am right now trying to find out which of the current 'must' artefacts are actually required by some regulations (banking is the functional area...) or by our CMMI certification, or which 'musts' are actually homegrown because someone once thought it would be a good idea to have time (in waterfall anyway). So I definitely agree when you say this is one of the difficult tasks... So far my understanding is that it still makes sense to perform some RE upfront, e.g. by defining Use Cases with maybe just the rough basic flow, to serve as a basis for further detailing in the SCRUM sprints. I also see some concepts that need to be elaborated upfront because they affect functionality that will be developed in different SCRUM teams eventually. I think about either establishing traces from these use cases so user stories eventually, or to refine the Use Cases at the end of the sprints/develop ment to make sure they reflect the reality - in order to have a sound basis for future maintenance. I am interested in your opinion and experience on this.
Reply | Reply with quote | Quote
 
 
0 # Marc Thibault 2011-10-26 01:05
Agile or otherwise, what it comes down to is either I stop doing what I'm paid to do and instead spend full time managing the development, or I provide a description of the product that's sufficiently clear that no one can possibly misunderstand what's to be built. That description is usually called 'requirements.' 'Formal Requirements' are requirements subjected to discipline and quality control. Whether or not they're appropriate is a function of how important the result is.
Reply | Reply with quote | Quote
 
 
0 # cindy wilson 2011-10-28 06:47
I was amused at your reference that ours is a "community accustomed to a lot of formal documentation." When I came to my current position, we practically had no documentation and considered it a huge achievement that we now know what our systems are supposed to be doing. There are many consumers of formal requirements documentation, not merely developers and testers. Often they are a source of information for users facing a change in operations. They are a one-stop shop for system support and a source of vital information for later analysts and developers who inherit the advancement and maintenance of an application. I f application requirements are spread out over the projects that impact those applications, especially for companies operating in a highly interfaced environment, where is the one final word on what the application is supposed to be doing, in a holistic sense? As a [user] I need [desire] so I can [rationale] doesn't supply that information in any complete and versioned manner and, in my experience is a cosmetic distraction that is then followed with the real requirement. Those needing to know what a system is supposed to do shouldn't have to search through years of stories that require change to an application, put them into chronological order and then read through enhancement history to find out what This System should be doing in This Case right now. Holistic system requirements provide several communities with a complete picture of the current requirements of a system. Whether or not they be critical for the development of the next enhancement is beside the point. They still have myriad uses and benefits for companies that choose - or are required - to maintain them. They do cost a little more to maintain, but I think in diminishing the documentation of requirements as part of the Agile process, the agile purists are short-sighted. Such documentation should be maintained and can be part of an Agile process.
Reply | Reply with quote | Quote
 
 
0 # Colin 2011-10-30 08:55
I agree with Cindy's comments above. In addition, I would add that good quality documentation pays for itself over the full life of the business system. Any system with a lifespan of several years is almost certain to need modifications / enhancements, to reflect legislation or changes in the business environment - long after the project has been closed down and the project members moved on.
Reply | Reply with quote | Quote
 
 
0 # Kurt Solarte 2011-10-30 10:53
@Yonico, I have found that a good place to start is by getting a good understanding of who (if anyone) is consuming the formal documents that are being produced. Once you know them, you have your key stakeholders. Now do what you do best, some business analysis to understand what each of them needs out of documentation. From there you can begin to discuss other forms of documentation. @Kurt McKeown, you have hit it on the head. SCRUM is a construction method, not a delivery method. SCRUM doesn't mention much of how the ever famous backlog gets filled up ;-). So yes I would encourage some upfront RE (typically light weight). Use Case are good, I also find modelling things to be very helpful. The idea is to get consensus on the vision and goals of the project before beginning. Also it is a very good idea to create traceability links between those early artefacts and the stories that will develop throughout the sprints. For you compliance challenges you will probably find it important to do some templating and automation so that your required doc can easily be created from your working artefacts (once that have been completed and baselined). Otherwise you may spend a lot of additional man hours formulating those documents out of your execution artefacts by hand. @Marc Thibault, I definitely understand the sentiment, but I disagree that it has to be a binary decision. I think we can make ourselves understood in better ways then by spending countless hours on formal documentation. This article isn't about 'Formal Requirements' it is about 'Formal Requirements Documentation'. I think we would all agree requirements should have some level of discipline and quality, the point of contention I find often is: does discipline and quality really mandate formal documents?
Reply | Reply with quote | Quote
 
 
0 # Kurt Solarte 2011-10-30 11:14
@cindy wilson & Colin, while I understand that certain companies or teams might be running in an ad hoc fashion (or feral as they say here is Oz), I have found our industry, that of Business Analysis and Requirements Engineering to be pretty document centric. I completely agree with your sentiment that Maintenance & Support Teams (among others) need proper understanding and a holistic view of systems; I disagree that the holistic view can not be achieved with Agile documentation as a part of the solution. I feel that what you are describing falls much more into the context of Enterprise Architecture (EA), which is an important aspect of systems development and can/should fit into agile project execution. If the EA artefacts are accurate (and updated), and the system was constructed using the EA artefacts and User Stories, I don't see anything that prevents the system from being maintained using the EA artefacts and User Stories. In large systems development, this is where moving off the whiteboard and sticky notes and into enterprise Application Lifecycle Management tooling becomes important. I think a good EA Model and Business Process Flow, coupled with a User Story and all of the comments, feedback, and approval that the User Story went through, tends to give a VERY good picture of that system, and usually without having to read a 200 page BRS.
Reply | Reply with quote | Quote
 
 
0 # Rajiv (Max) Roy 2011-11-07 15:31
@Kurt Mckeown - agree with you. You cannot start off doing an Agile project on Day 1, so to speak - that leads to chaos. You need to establish several parts of the solution, such as > SAD showing the 'helicopter view' of the solution, especially >> Defining its interfaces to existing & new systems >> Component/patte rn architecture & re-usages >> Baseline Project Plan of team activities >> Foreseeable Issues & Risks >> Production Release & UAT Plan > Stakeholder Expectations in high-level BRS - perhaps via Use Cases. > Data & Process Models I believe it is an art form when to stop these Upper - SDLC phases without trying to lock in the user too early. There should be enough leeway for the Agile Iterations to provide the application 'prototyping' & user experience and be able to adjust & cater to the User Stories and functional enhancements coming out of each iteration. And,yes, documentation at the end of each Iteration is VERY important.
Reply | Reply with quote | Quote
 

Add comment