Skip to main content

Author: Kurt Solarte

How Agile Requirements and Project Documentation Live Together

Just In Time and Fit for Purpose

Agile developers work under the assumption that uncertainty is so common and change so large that no upfront architecture and design can possibly be exactly right or fit for purpose further down the line. Therefore, development is iterative and as simple as possible, with a refactoring approach to architecture and design.

Agile development teams start with an idea, break it down into features and functions, write high-level requirements and then get started. The objectives are to document just enough to proceed with construction, complete a select set of the highest-priority requirements in each sprint, collect requirements in a backlog and produce documentation that anyone can use when they are working with the software.

Having documents that anyone can use is a laudable goal; however, in some cases, a document that is supposed to be fit for everyone ends up being fit for no one.

The Maintenance Documentation Gap

Many business-critical projects, such as commerce applications or portals, are now agile. In these projects, it’s common to have an outside agile development shop creating the application. The shop works on the project for 12–18 months and then it is turned over, either to the company that needs the application or another vendor, for maintenance.

Although the lightweight stories and requirement artifacts work really well for construction, they can’t just be handed to maintenance as is. These stories and artifacts do not have the information needed to maintain the application for a number of reasons.

Often, when a team gets in the middle of a big project, things change and those changes might not make it back into the documents. The attitude is that the software has already been delivered, and it’s time to move on to the next sprint. So, maintenance ends up with out-of-date documentation. In addition, agile documents are often very minimalist without the overview often needed by a maintenance team.

The Audit Documentation Gap

In some industries, there are often outside rules and regulations that applications must comply with. Because traceability is important in large projects, large agile teams often use tools to link artifacts for traceability, but at times, these are not sufficient.

For example, I worked on a big portal implementation for a pharmaceutical company. It was an internal portal that involved data from drug studies. In that situation, not only was the data access subject to audit, but also the construction of the data access portal itself. Although we linked artifacts, we were very focused on developing the application and website, and there was a scramble for documents and required traceability when the auditors showed up.

Like maintenance documentation, the information auditor’s need is not necessarily found in “one-size-fits-all” requirements documentation. Attaching traceability to the project is also not sufficient. Auditors need documentation that demonstrates compliance.

Agile Documentation

Closing the gap between the documentation produced for agile projects and the needs of maintenance teams and auditors can be achieved by taking the same approach to delivering documentation as that taken for delivering code. Instead of producing documentation that everyone should be able to use but can’t, think about providing maintenance and audit documentation as separate deliverables.

In the projects I work on, we decouple requirements, stories and artifacts and create other deliverables that are part of the sprints. For maintenance, we build some frameworks based on what maintenance wants. We harvest things out of agile requirements and then look at what gaps exist. Part of the sprint is filling in the gaps.

For audit documentation, we get the full requirements and make them part of the story. In fact, the story is not complete until we have run the report and compliance matrices and connected them to the story. We recognize that we might have to add more traceability, that compliance and audit regulations are continually evolving and emerging, and that we need to keep up.

Coexistence

Requirements, artifacts, maintenance documentation and audit documents can all coexist, not necessarily in the same location, but as part of different sprints. And when you take the just-in-time and fit-for-purpose approaches, all of which include taking advantage of reusable parts when possible, generating documentation is easier.

After all, when you look at maintenance manuals or administration guides, you can see that they contain some elements found in design documents. Some of the traceability required by auditors is similar to what agile project teams want, such as links between business need, development work item and acceptance tests.

In the agile projects I work on, we identify these similarities, publishing out a shell and building a template with an outline that includes an extract from the business process flow or architectural diagram. Some of the objects we create in other tools can be inserted in the manual. The result of this approach is “mini-ocuments.” Our mini-documents for maintenance have been more popular than even the older large design documents because maintenance teams can move right to the space where a defect is detected. They no longer have to deal with huge documents that are difficult to navigate.

Conclusion

Documentation for applications built by agile development teams that are based on requirements, artifacts and stories can come up short when they are needed for other purposes such as maintenance or audits. Providing documentation that suits different purposes is possible if you take an agile approach. Determine documentation needs upfront and include the different documentation builds in your sprints and you will provide the right documentation solution for all stakeholders, including maintenance and audits.

Don’t forget to leave your comments below!


Kurt Solarte is a Managing Consultant working for IBM Australia in Sydney; focusing on Agile Development methods. Kurt recently spent eight years with IBM Global Business Services in the U.Ss as a Sr. Business Analyst; where he specialized in keeping business analysis alive and relevant in the agile delivery of eCommerce, web portal, and business analytics projects.

Does Formal Requirements Documentation Have a Place in Agile?

The question above is one I have faced many times, and is usually one of the first questions Business Analysts ask when confronted with an agile method. Many times this comes not so much as a question, but as a statement like, “This agile method must incorporate our documentation.” Whether this sentiment comes as a question or a statement, my answer tends to be the same: It can fit, if it has to fit.

To be certain, Formal Requirements Documents do not have a place in all, or even most, agile environments. However, just because an organization has a non-negotiable obligation to produce Formal Requirements Documents doesn’t mean agile delivery methods are not an option for that team. While Formal Requirements Documents, like Business Requirements Specifications (BRS) or Business Requirements Documents (BRD), seem to be the opposite of agile, they are often a required part of doing business. Oftentimes this is in regulated environments, such as companies that have certain documentation as part of an ISO or CMMI certification that has been achieved, or consulting firms with contractual obligations for documentation.

The path to integrating Formal Requirements Documents into agile methods includes:

  • Comprehending the underlying request or regulation for Formal Requirements Documents
  • Performing a gap-analysis of desired agile work products and required formal work products
  • Leveraging templates and automation tools to ease the publication of Formal Requirements Documents

The above steps provide a good framework for integrating agile and formal documentation. In reality, the first step is finding out if Formal Requirements Documentation really HAS to be produced. Although many demands for Formal Requirements Documentation are valid, many others are simply the knee-jerk response of a community accustomed to a lot of formal documentation. This is often one of the more difficult tasks in the process, and requires a touch of process analysis, method knowledge and a knack for organizational change management. However, once the requirement for Formal Requirements Documentation has been validated, I assure the team our new process will result in the required documentation and then immediately take the focus away from document-centric thinking. As with most agile engagements, the focus should be on requirements methods that capture what is needed to build a shared understanding; including the use of iteratively developed user stories and inclusive modelling techniques that enable active stakeholder participation.

Throughout the process of understanding the regulation or contract that is dictating the Formal Requirements Documentation, one will typically find the regulations rarely dictate how or when to capture the requirements, the level of traceability detail required or mandate formal reviews. Given these facts, there is obviously still plenty of room for agile methods within a formal framework. It is also important to get an understanding of which desired agile requirements components will comprise the Formal Requirements Documentation, and determine if there is a gap between the desired agile requirement artefacts and the regulatory constraints. Often additional artefacts will be needed, such as business rules or requirements pertaining to sensitive data. However, comprehension of the regulation is particularly critical in this area, as these additions are typically dependent on the risk or profile of the project. Typical examples might be User Stories (or Light Weight Use Cases), high-level business process sketches, and User Interface prototypes — all of which will be created in an agile and highly collaborative manner with the team and business stakeholders. Additionally, there might be mandated security rules or requirements pertaining to sensitive personal data that are typically created by an analyst based on industry specifications or standards.

Once a team has decided what its method will be, and completed a gap-analysis to confirm all the documentation needs are covered by the new method and work products, the next step is to look at how the final document will come together. This planning step is essential prior to the project kickoff so the Business Analysts and Delivery Teams will be able to focus on the design and build of the project rather than worry about the formation of documentation. The creation of lightweight templates and traceability matrices will go a long way toward ensuring a clean and compliant document can be produced from the agile work products. Many thought leaders in the space strongly advocate for templates, like Mike Cohn’s widely used User Story Template: “As a [end user role], I want [the desire] so that [the rationale].” Beyond the traditional benefits these templates give a team, they also become very important in the final assembly and publication of formal documentation.

A vital component to creating Formal Requirements Documentation from an agile project is automation for requirements traceability, version control and document publishing. There are many tools on the market that can assist with this, or custom applications or scripts can be used. Without some sort of automation, creating the Formal Requirements Documentation will become an onerous manual process. The proper level of automation will allow a team to quickly extract, assemble and publish a formal document from a collection of agile and mandated work products. The ease and quickness of automation is what allows the formal document publication to keep pace with the agile method — allowing for publication of new formal documents as often as the team sees fit or the regulations mandate. A typical scenario is to produce a formal document at the beginning and end of every sprint in order to capture the normal changes to requirements and prioritization that takes place during an agile sprint.

In summary, Formal Requirements Documentation can absolutely be integrated into the agile process, becoming just another by-product of an agile delivery team.

One good technique to achieve this is to:

  • Gain deep understanding of the regulations
  • Perform a gap-analysis of the agile work products and the mandated documents
  • Use templates and automation for traceability matrices and document publishing

While this type of hybrid method is often frowned on by some ‘agile purists,’ I find it extremely agile to acknowledge that one process does not fit all, and a bit of flexibility is usually the answer.

Don’t forget to leave your comments below.


Kurt Solarte is a Managing Consultant working for IBM Australia in Sydney; focusing on Agile Development methods. Kurt recently spent eight years with IBM Global Business Services in the U.Ss as a Sr. Business Analyst; where he specialized in keeping business analysis alive and relevant in the agile delivery of eCommerce, web portal, and business analytics projects.

Three Baby Steps Toward Agile Requirements Management

FeatureAug16thIf you are beginning to consider a move to more agile methods, you may be looking for a set of ‘best practices’ for agile requirements elicitation and agile requirements management. What you will likely find is many prescriptive and detailed ‘best practices,’ which I have personally found to be situational at best, and often just too much to consume for an organization new to the concepts. What I propose to offer in this post are a few ‘baby steps’ that can help an organization move toward an agile environment and prepare for some of the more prescriptive methods like SCRUM or eXtreme programming (XP).

When making a move toward agile methods, there are three main recommendations that I believe should be considered to prepare for a more agile requirements practice, and a team as a whole.

  1. Collaboration ─ While this is obviously a good idea in all methods of requirements management and all stages of a project lifecycle, it is an essential element to agile methods and agile requirements management.
  2. Embrace Change ─ This is almost a counter-intuitive step for many business analysts. To be agile in requirements management, we must acknowledge change happens, embrace it, and make it part of our process.
  3. Support of upper management ─ Becoming agile is an organizational change as much as a development practice, and without clear support from upper management, this change will be difficult, if not impossible.

Recommending good collaboration is a bit like recommending good work ethic; it seems as if it should go without saying. However, for organizations used to a very siloed waterfall method, real collaboration as a standard operating procedure may have fallen by the wayside. It can become easy in a waterfall method to simply over-document deliverables and blindly send them to the next team. In an agile method, an organization will begin to keep ‘living documents’ that constantly change and grow ─ this requires true collaboration and teaming. In order to include all members of the project team and stakeholders properly, an organization must not only take on more inclusive methods, but must educate all associated people on the use of the method, including management and stakeholders. If one expects individuals to be involved, the individuals must know how, why, and when they are expected be involved. Simply said, to create an inclusive method, all individuals associated with the project must feel and be encouraged to be involved. One simple step many larger organizations have found useful is the adoption of stakeholders’ terminology. Often in agile methods, we get very focused on the new names, terms, and roles ─ all at the detriment to collaboration. Many stakeholders will feel more comfortable with requirements, time periods, and project roles being expressed in their own terminology. This should be seen by the project team as a minor secession for the sake of greater collaboration.

Embrace Change. To many Business Analysts and Project Managers, this will seem counter-intuitive. As professional BAs and PMs, we have been taught to identify all requirements and lock down the scope. However, we all know change does happen, and what often defines the success of a project is how we deal with that change. Traditionally, an organization would manage this change with some sort of project or scope change management process, which gives a structure to altering requirements that are typically fully elicited. In agile methods, organizations acknowledge that change is constant in technology projects, and move to embrace that change. To make steps toward agility, an organization may need to change their requirements elicitation methods. The elicitation can begin with very broad requirements that outline a general scope, do some high-level requirements modelling, prioritize what has been captured, and prepare the team to change as needed. Once the broad requirements are documented and ranked, the team can then begin to add further detail to requirements in an ordered fashion. This means the stakeholders will give detail as it is needed, and the team will change documentation as details emerge. The stakeholders know their priorities and requirements will change, and the project team knows the project documentation is ‘alive’ and changing as the project moves on, and as the need for detail requires.

While the above-detailed steps are important ways to ease into agile, arguably the most necessary step is getting upper management support. The management lines for both the project teams and the stakeholders must understand the use of agile methods is an organizational change, and not just a development method. These management lines must fully comprehend the concepts behind embracing change, increasing collaboration, and the notion of practicing constant requirements prioritization. As with many successful organizational changes, the use of ‘on the ground’ champions of the new process are important, but a clear message of support by company leadership is imperative. A project kick-off that includes an executive saying a few words of support for the new methods and mentioning that management is watching for this to be a successful implementation will go a long way to drown out the voices of the detractors.

As an organization strives to become agile, it is important to remember to be agile in the implementation of agile. Meaning, only take on as much agile process and change as your organization can handle in each ‘sprint,’ or short change window; and only keep the agile processes needed to improve implementation of technology projects. Organizations can often lose focus of their underlying purpose for adopting agile methods, and begin to focus on ‘becoming agile.’ There is not a prize or reward that comes with completely adopting any particular agile method, so look to embrace the parts of agile methods you need to find your success, and question anything else — because that is truly becoming agile.

Don’t forget to leave your comments below.


Kurt Solarte is a Managing Consultant working for IBM Australia in Sydney; focusing on Agile Development methods. Kurt recently spent eight years with IBM Global Business Services in the U.Ss as a Sr. Business Analyst; where he specialized in keeping business analysis alive and relevant in the agile delivery of eCommerce, web portal, and business analytics projects.