Requirements Review Challenges

Any time someone other than the author of a work product examines the product for problems, a peer review is taking place. Reviewing requirements is a powerful technique for identifying ambiguous or unverifiable requirements, requirements that aren’t defined clearly enough for design to begin, conflicting requirements, and other problems. The most effective type of peer review is a structured and systematic examination called an inspection. Inspection of requirements documents is one of the highest-leverage software quality techniques available. Several companies have avoided as many as 10 hours of labor for every hour they invested in inspecting requirements documents and other software deliverables. A 1,000 percent return on investment is not to be sneezed at.

A peer review is both a technical activity and a social activity. Asking some colleagues to tell you what’s wrong with your work is a learned—not instinctive—behavior. It takes time for a software organization to instill peer reviews into its culture. This article points out some common challenges that organizations face regarding requirements reviews, with suggestions for how to address each one.

Large requirements documents. The prospect of thoroughly examining a several-hundred-page requirements document is daunting. You might be tempted to skip the review entirely and just proceed with construction—not a wise choice. Even given a document of moderate size, all reviewers might carefully examine the first part and a few stalwarts will study the middle, but it’s unlikely that anyone will look at the last part.

To avoid overwhelming the review team, perform incremental reviews throughout requirements development. Use inspections to take a close look at high-risk areas, and use informal reviews for less risky material. Ask particular reviewers to start at different locations in the document to make certain that fresh eyes have looked at every page. To judge whether you really need to inspect the entire specification, examine a representative sample. The number and types of errors you find will help you determine whether investing in a full inspection is likely to pay off.

Large inspection teams. Many project participants and customers hold a stake in the requirements, so you might have a long list of potential participants for requirements inspections. However, large review teams increase the cost of the review, make it hard to schedule meetings, and have difficulty reaching agreement on issues. I once participated in a meeting with 13 other inspectors. Fourteen people cannot agree to leave a burning room, let alone agree on whether a particular requirement is correct. Try the following approaches to deal with a potentially large inspection team:

• Make sure every participant is there to find defects, not to be educated or to protect a position.
• Understand which perspective (such as customer, developer, or tester) each inspector represents. Several people who represent the same community can pool their input and send just one representative to the inspection meeting.
• Establish several small teams to inspect the requirements in parallel and combine their defect lists, removing any duplicates. Several studies have shown that multiple inspection teams find more requirements defects than does a single large group. The results of parallel inspections are primarily additive rather than redundant.

Geographically separated reviewers. Organizations often build products through the collaboration of geographically dispersed teams. This makes reviews more challenging. Teleconferencing doesn’t reveal the body language and expressions of other reviewers like a face-to-face meeting does, but videoconferencing can be an effective solution. Web conferencing tools allow reviewers to ensure that they are all looking at the same material during the discussion.

Reviews of an electronic document placed in a shared network repository provide an alternative to a traditional review meeting. In this approach, reviewers use word-processor features to insert their comments into the text. (This is how we reviewed each other’s work as we were writing our book) Each comment is labeled with the reviewer’s initials, and each reviewer can see what previous reviewers had to say. Web-based collaboration tools also can help. Some requirements management tools include components to facilitate distributed asynchronous reviews that do not involve live meetings. If you choose not to hold a meeting, recognize that this can reduce a review’s effectiveness, but it’s certainly better than not performing the review at all.

Unprepared reviewers. One of the prerequisites to a formal review meeting is that the participants have examined the material being reviewed ahead of time, individually identifying their initial sets of issues. Without this preparation, you risk people spending the meeting time doing all of their thinking on the spot and likely missing many important issues. In fact, if you are invited to participate in a requirements review and do not have adequate time to go over the material in advance on your own, don’t even bother to go to the meeting. It’s a waste of everyone’s time.

One project had a 50-page SRS to be reviewed by 15 people (far too large to be effective and efficient). Everyone had one week to review the document on their own and send issues back to the author. Not surprisingly, most people didn’t look at it at all. So the lead BA scheduled a mandatory meeting for the inspectors to review the document together. He projected the SRS on the screen, dimmed the lights, and began reading through the requirements one by one. (The room had one very bright light shining down in the middle, directly on the lead BA—talk about being in the spotlight!) A couple of hours into the review meeting, the participants were yawning, their attention fading. Not surprisingly, the rate of issue detection decreased. Everyone longed for the meeting to end. This BA let the participants leave, suggesting they review the document on their own time to speed up the next review meeting. Sure enough, being bored during the meeting triggered them to do their prep work.

Reviewing a requirements document is not everyone’s idea of a fun time. Peer reviews can be tedious and time consuming. Nonetheless, if you’re serious about maximizing the quality of your software, your teams will review all of their requirements, using inspections to carefully examine the most critical or error-prone portions. The teams I know that have adopted requirements inspections agree that every minute they spent was worthwhile. Keep the suggestions in this article in mind to help you maximize the return on investment you make in requirements reviews.

Don’t forget to leave your comments below.

Author Bios:

Joy Beatty is a Vice President at Seilevel. Karl Wiegers is Principal Consultant at Process Impact. Karl and Joy are co-authors of the new book Software Requirements, 3rd Edition (Microsoft Press, 2013), from which this article is adapted.