Bad Ass BA; Peer Review. Part 3.

Co-authored by Rebecca Burgess

Use and Profit from Peer Reviews of Requirements Documents

This article is part 3 of a 4-part series on the topic of peer review of a requirements document. Before we dive into part 3, indulge me in a personal note.

My organization recently changed from having a peer BA review another BA's BRD to a novel process in which four BA peers review one BRD. Each of the four peers has a different perspective based on their professional experience. Two weeks ago one of my BRDs went through peer review. In our process, first the reviewers provide their written comments. Then the BRD author has a full day to review the comments before conducting a structured walk-through of the BRD with all the peers in a conference call.

While reviewing the combined comments on my BRD, I experienced a series of emotions I haven't felt in quite a while: initial horror, shame, indignation, despair and dismay. One of my peers wrote me a note, "Hey Cecilie, is this a test? I'm commenting on your BRD with words that I hear in my head in your voice." I wish I could say that it was a test, but it wasn't. I had submitted a BRD to the peer review process under duress (time pressure); the BRD wasn't my best work but I honestly didn't expect that scrub cloths would come back so dirty after my peers had examined my work.

I spent several hours cleaning up that BRD. I wasn't looking forward to the conference call review where, in accordance with our internal process, I would walk the group through the comments that were marked "High" and "Showstopper" - I would have to reply to each such comment and offer a suggestion for remediation. All High and Showstoppers had to be resolved, otherwise my BRD would fail the peer review and fail to move to the next step of the BRD approval process.

My BRD received a conditional approval from my peers - I had to get a couple of questions answered from the stakeholders first. Fortunately the answers came back quickly. Another hour to tidy everything up, and the BRD was fully approved by the peer group to move forward. Most of the time I do leave my ego at the door but apparently I kept it with me during the review because it took a couple of hours of mentally pouring honey on my bruised ego for me to recover.

We've been running this peer-group review process for about two months. We are hearing from the downstream reviewers - all managers - that the quality of the BRDs has improved noticeably. I can attest that the quality of my BRD benefited from the scrutiny of my peers. The difficulty in a peer review is that, if the peer reviewer is also your friend, your friend may not be able to be as honest as you need him/her to be. The team of four gave me honest, constructive comments about scope and risk; they caught inconsistencies in the requirements, and questioned the testability of some of the requirements. An un-anticipated side effect of this peer-group review process is that we are learning from each other about how to address BRD authoring problems much faster than anyone expected.

So, that was a long note, but I hope it is helpful to anyone who is nervous about giving peer review a try. Embarrassment will not kill you and in the end you will be more proud of your work.

Let's do a recap before we begin part 3.

One way to conduct a review is as a series of go/no go checkpoints.

Just to review, here are the four checks:

Business Case Synchronicity Check
Requirements Document Sanity Check
Requirements Statements Content Check (In this issue)
Requirements Document Housekeeping Check (December 2009)

And here are the typical excuses not to do a peer review, and why those excuses are bogus.

It's scary! - They won't say this out loud, but it's true. You must separate your self esteem from the evaluation of your work by learning to accept criticism as constructive and valued, rather than destructive and to be avoided at all cost.
It takes time - Yes, but the payback is substantial, both in minimizing the impact of errors, and in maturing the BAs on the team.
We are already behind schedule - Would you rather let the customer catch the errors? Or worse yet, miss the errors?
I don't know anything about that project - So you'll limit your review to the more general, BA-level comments; that's still valuable.
It's hard to do - True. The more you practice, the less difficult it will be the next time.

In the first article we checked for Business Case Synchronicity. We found that the BRD synched up with the business case, that is, benefits for doing the project have been identified, there is an expectation of Return on Investment, and there is an expectation that the project will report on progress towards achieving the ROI claim. Moreover the customer has identified the means for validating project success.

In the second article we performed the Sanity Check. We verified that the Scope not only identifies what is in scope, but what is out of scope as well. The document has a business context diagram, a 30,000 foot view of what the project is about in terms of system functionality, the business entities that will interact with the system, and information that is exchanged between the entities and the system. Finally we verified that the categories of the requirements made sense for this particular BRD.

Now it is time to look at the requirements statements themselves, both individually and, as they relate to each other.

Requirements Statements Content Check

Take a deep breath, hold it, and let it out slowly. Make sure you aren't going to be interrupted for at least half an hour. Depending on the length of the document, you'll need chunks of uninterrupted time to do this part of the review.

Check the box next to the item if you can answer 'yes' for the document you are reviewing.

Clarity

Is each requirement relevant to the business problem?

Are all requirements actually requirements, not design or implementation statements?

Design and implementation statements are "how" statements; a requirements document has "what" statements that ideally do not imply a desired solution.

Do you understand each requirement? Does each requirement make sense to you?

If a requirement doesn't pass your "sniff test", it likely won't make sense to anyone else.

Are the requirements "solution agnostic", i.e., do they convey the business need without specifying a solution?

Are the statements constructed as proper statements or sentences?

If you think the requirement statement is a fragmented thought, other people will too. The BA author may need to request more information (not necessarily more detail).

Is the content in each statement just sufficient to convey the idea?

Statements can become too terse (fragments), or too verbose. "Verbose" can be any of these:

Too wordy, but the statement is still just one requirement
More than three lines long.

A statement that is longer than three lines of text is not always verbose, but verify!

Are the business requirements written in non-technical language?

Is the text understandable to a business person and understandable by an independent third-party? The problem isn't just a missing term in the glossary; if the requirement is jargon infested, the BA author should revise it for the benefit of mere mortals.

If a requirement needs specific business knowledge to understand, will the people who implement and/or test the requirement have that knowledge?

If not, the BA author may need to provide more information in the requirement, or explanatory text along with the requirement.

Are the requirements constructed to reduce the possibility of more than one interpretation?

If not, does the requirement need more detail to limit it to a single interpretation, or should there be a set of related requirements to reduce the likelihood of ambiguity?

Does each requirement address a single concept, topic, element, or value?

It takes a clever BA to decompile a complex idea and turn it into a requirement. The result of the first attempt may be an overloaded requirement, i.e., a single requirement that is actually more than one requirement. Look for requirements that have more than one relative clause, more than one instance of a semicolon, or the frequent red flag, more than three lines of text. These attributes are often indicators that the requirement needs to be split up into a set of multiple-but-related requirements, each with their own identifier, e.g., Security-011a, Security-011b.

Are requirements statements distinguishable from the explanation that often follows a requirement?

Sometimes a requirement needs an explanatory note, that is, text that is supplementary to the core requirement. Be sure that the note is separated, even by a blank line, from the core requirement so that it is clear which statement is the requirement and which is the explanatory note. Notes should not have the telltale "shall" verb.
Sometimes the explanation is a design constraint. Design constraints need to be assembled in their own section such as an appendix section for design constraints, or, at the very least, listed in the Assumptions.

Testable, Verifiable

Are the requirements verifiable by testing, demonstration, review, or analysis?

Do the requirements have success criteria, and a metric that specifies the success threshold?

"I'll know when I'm happy" is not an acceptable evaluation criterion!

Key, high priority requirements need explicit success criteria and/or metrics. Lower priority requirements or very detailed requirements may have implicit success criteria.

Are the criteria for a "pass/fail" evaluation clear and quantifiable?

An example of a clear evaluation criterion is a named process that a human or a machine can execute to verify the requirement has been met.

If the success criteria/metrics for a requirement are known, is that information associated directly with the requirement?

Sometimes the success criteria/metrics get separated from the requirement, e.g., in another section of the document. Use cross-references or some other means for the reader to put the two together.

Are the requirements free from weasel words? (see BA Times article http://www.batimes.com/component/content/article/106-articles/377-bad-ass-ba-caution.html)

Weasel words will be the ruin of the project.

Business-Critical Dates

Are timing constraints such as due dates, deadlines, and completion-date dependencies identified?

Are the "critical" dates truly critical?

There are many ways to define "critical to the business". It is important to distinguish "critical" and "important". In my humble opinion, there are four types of truly "critical" dates:

Beyond this date there will be an impact on revenue generation
Beyond this date we will lose the ability to conduct business
If we miss this date (which we announced to the media) we will be subject to ridicule and derision in the press which will drive our stock price into the ground
If we miss this date we will be subject to contractual or regulatory sanctions, such as a monetary fine. For example, being out-of-compliance, or a real estate lease date expiring, or a contractual completion date not being achieved.

Are the "important to customer" dates articulated?

Even if the date isn't "critical" by the definition above, a date can be important to the customer and should be identified as such so that the project manager knows about both the date and the expectation.

3rd Parties and Partnerships

Are the major requirements identifying the need for information to be shared, processed, or stored by 3^rd party partners or hosted applications?

Compliance

Are compliance requirements identified?

Compliance requirements often first appear as an assumption. Assumptions can be overlooked during design and implementation. Calling out a compliance requirement ensures that the project test plan will have a line item for verification of compliance. Examples of compliance are data privacy (Personal, Identifying Information (PII)), payment card industry (PCI), legal, financial, and governing entity policy compliance.

If the project does not have any compliance requirements or impact, is that made explicit?

Adding the statement, "This project do not have any compliance requirements." is better than just leaving the Compliance section of the document blank, which looks like the author forgot something.

Business Communication Processes

Are the major requirements related to supporting business communication processes included?

If the project is going to have an impact on how users do their work, in terms of what they see or what they do, then there needs to be a communications plan. The BA author may think that it is overkill to have a requirement for a standard project deliverable but the manager of the group that provides end user support may see things differently.

Internationalization and Localization

Are the major requirements related to internationalization and localization included?

Internationalization, known as I18n (meaning "I - eighteen letters -N") to people who do this for a living, is the process of planning and implementing products and services so that they can easily be adapted to specific local languages and cultures. Doing the work of translating the text into local language and culture is called localization, aka "L10n".

Timing / Response

If time-critical functionality is known when the requirement is written, is the time-criticality identified, and are timing criteria specified with metrics?

A requirement for "fast response" isn't testable! "Fast" compared to what? Is less than 10 seconds acceptable? Is less than 2 seconds acceptable?

Reporting

Is there a requirement for creating a report to show progress towards achieving the projected return on investment (ROI) on the project/product?

According to best practices, a requirements document should have a business case. That business case identifies the expected ROI for the work that will be done. For the project to be truly successful, it needs to produce the evidence for delivering on the ROI claim.

Are the reports/dashboards/extracts that are expected from this project identified?

Are the audiences for those reports/dashboards/extracts identified?

Unknown, Missing, Incomplete Requirements

Are requirements where information is unknown, missing, incomplete or "to be determined" (tbd), annotated with a reason for the incompleteness?

Some BRD templates have an "Open Issues" section were unknown, missing, incomplete or "still under discussion" topics are called out. Ideally each issue has the following information to help the BA close the issue:

A unique identifier for the Issue, this could be as simple as "Issue.1"
A description of what the issue is
The date the issue was first identified
The name of the person responsible for resolving the issue and a target date for that resolution
A status for the issue, e.g., Open/Closed/Deferred. Seeing a column of "closed" issues is so satisfying.

Requirements Statement Style

Do the requirements statements use one verb consistently, e.g., shall or will?

Are the requirements articulated at a consistent level of detail?

Should any particular requirement be specified in more detail? In less detail? It is important that each requirement be stated at an appropriate level of detail, and that all the requirements in a given document be at the same level of detail.

Is level of information in the document consistent with commonly accepted practices for your company?

Each company has its own rules for the level of information that should be included in a requirements document. If your company doesn't have a guideline, here is a possible starting point:

A Business Requirements Document has project-level requirements that describe the needs of the Stakeholders.
A Functional/Non-Functional Requirements document has requirements at the software system level and may include requirements at the subsystem-level.
A System Requirements document has even more detailed requirements.

Do the requirements statements use the "Trigger Actor Action Condition" syntax?

Requirements in the form of "Ability to ..." are ambiguous unless there is a context-setting statement that all requirements describe who or what the actor is. Requirements must identify the actor as a human or an automated agent. "Ability to ..." statements create problems because the reader has to guess at whom or what will have the ability.

Consistency of Requirements

Have inconsistencies, omissions or redundancy been identified and corrected?

Do requirements use consistent terminology to the same object?

It is normal for different groups within a company to have different terms for the same object. For example, in the United States what we call "regulations" may be called "rules" in Europe. It is important to identify both terms in the document's glossary. Use one term consistently in the requirements.

Have requirements that describe two or more actions that conflict been identified?

The most common types of conflict are logical conflict and temporal conflict.

An example of logical conflict is when there is a general requirement for a "green" system, specifically, no printing, and, a conflicting requirement for the automatic generation of the quarterly sales results report to be faxed to the executives in Japan. An example of temporal conflict is when one requirement says that customer satisfaction data will be sent to an FTP site on a weekly basis and another requirement says that customer satisfaction data will be uploaded and processed on a daily basis.

Once such requirements have been identified, there are two possible ways to proceed. Either have the stakeholders come to an agreement, or, create an Open Issue to address the conflict as part of the design effort.

Requirement Traceability

Are all requirements traceable to at least one "in scope" objective?

Do all objectives/success criteria map to at least one requirement?

If not, there may be a gap in the body of requirements.

Common problems with Categories of Requirements

Do the requirements statements in the categories actually belong in those categories?

UI requirements are really good at hiding in a category other than a User Interface category.
System Interface requirements are often distributed throughout a BRD. Assembling interface requirements in one place makes it so much easier to understand what a project need is. Or, if there is a good reason for not moving the interface requirements from their functional category, give the technical team a reason to appreciate the BA by suggesting a table of Interface cross-references so that at least there is a logical assembly.
Be sure to distinguish internal interfaces from external interfaces. In some companies interface requirements are documented in a separate Interface Requirements Document (IRD) or Interface Control Document (ICD)

Are scope statements, assumptions, business rules and role/responsibility statements in the right category?

Sometimes a statement begins life as a requirement but over the course of revisions, it evolves into something different such as scope, assumption, business rule, or role/responsibility statement.

If there are interface and integration requirements, are they in separate categories?

Business people often use the terms interchangeably. The technical team knows there is a big difference!

Did you answer "yes" to the all the questions? If so, do you now believe in your heart that the requirements are good? If not, mark the requirements that need work and identify what your concern is.

If one or more of these items is not checked, there is some risk that the requirements are inadequate. Your mission now is to convey these "defects" to the BA author in a kind manner. If you are expected to return written feedback, take time to explain what is missing, and provide examples, if possible. If you will be participating in a structured walkthrough meeting, make notes of what you want to say so that your comments are thoughtful and constructive.

The document should not be seen by the customer or stakeholders until the problems are fixed. If the requirements are good, you can now move on to the last quality check, the good housekeeping check.

This is the third of four articles addressing Use and Profit from Peer Reviews of Requirements Documents

Business Case Synchronicity Check
Requirements Document Sanity Check
Requirements Statements Content Check
Requirements Document Housekeeping Check [December, 2009]

Don't forget to leave your comments below

Cecilie Hoffman is a Senior Principal IT Business Analyst in the Business Analysis Center of Excellence, Symantec Services Group, Symantec Corporation. Cecilie's professional passion is to educate technical and business teams about the role of the business analyst, and to empower the business analysts themselves with tools, methods, strategies and confidence. Cecilie is a founding member of the Silicon Valley chapter of the IIBA. She writes a blog on her personal passion, motorcycle riding, at balsamfir.com. She can be reached at Cecilie_Hoffman@Symantec.com.

Rebecca Burgess is the Business Process Methodology Analyst in the Commerce Lifecycle Transformation Office at Symantec and a Certified Six Sigma Black Belt. After many years of uncovering problems and determining root causes in software, she is now applying her BA skills to strategic process design and improvement. She can be reached at Rebecca_Burgess@symantec.com.

Set as favorite

Bookmark

Email this

Comments (9) Add Comment

Subscribe to this comment's feed

...
written by drbonebrake, November 10, 2009

Thank you for this series and especially your relation of a personal experience. The key for the person whose document is under review is to recognize the opportunity for a learning experience. The benefit to the team is the ability to identify requirements ambiguity or errors while they are at the least expensive stage to correct.

Like you, over the years I have heard all the excuses as to why peer reviews of project artifacts cannot be done, but it is a sign of a strong and professional analyst to plan time for and invite peer reviews of their deliverables. In the end, such analysts will become stronger than their peers that duck the process. I agree, it can be difficult to emotionally separate one's self from their work and take the lumps when one is working an aggressive time line or with austere resources, but if we put things in perspective, “what doesn’t kill us, makes us stronger.” Thank you again for an excellent series.

"Requirements Analysis is a Team Sport"
Doug

...
written by rforsberg, November 10, 2009

A comment mostly related to the article title itself.

I find myself personally growing tired of the so-called Bad Ass BA techniques related articles that seem to get published again and again. Yes - I understand that the BA acronym matches nicely with "Bad Ass" but I have to admit this initially funny play on words has grown obvious and tiresome for me.

There is really nothing "Bad Ass"ed about the discipline of BA. In fact the discipline is fairly risk adverse and conservative by nature. I have never felt like a bad ass planning for requirements elicitation, never felt like a bad ass during requirements analysis and elaboration - certainly never felt like a bad ass validating a solution.

Please, it's predictable and growing tiresome. So...

...
written by cecilie.hoffman, November 10, 2009

Doug, I love your by line, "requirements analysis is a team sport" - so true! My co-author and I used to be each other's peer reviewers. When she moved to a different organization, I lost the only person I trusted to do an objective review. In our BA CoE, objectivity results from multiple pairs of eyes on a single BRD. We rotate reviewers every six weeks so everyone has a chance to recognize what they have learned from when their document was reviewed. Thanks for leaving a comment!

...
written by jnunemacher, November 10, 2009

For a few years I have been the sole analyst around a development team. I am looking forward to joining a team with a peer to collaborate with. Especially with high impact project, it is hard to imagine that any BAs work would go to the customer without some sort of internal sanity check - if not a full on review. Yes, the first few times are hard, but as you say, with practice it should get easier (for the reviewer and the reviewee). All that being said, I could almost feel my face flush along with yours as your ego was bruised and battered through your review. :)

...
written by cecilie.hoffman, November 10, 2009

Rforsberg - I understand your point, there is nothing inherently "bad ass" about doing business analysis. Moreover, thanks to the work of the IIBA, the profession now has standards and best practices, so in the future, taking risks that are "just doing my job" shouldn't seem like such a big deal. So, from a marketing perspective, has the "Bad Ass" brand name run its course? I'm sincere in asking for your input, and, the input of others who are reading this.

...
written by rebecca_burgess, November 10, 2009

rforsbert said:
"There is really nothing 'Bad Ass'ed about the discipline
of BA. In fact, the discipline is fairly risk [averse] and
conservative by nature...."

I really must respectfully disagree. The purpose of the BA discipline is to decrease risk _to_the_company_, but my personal sense of hazard can feel/be quite high.

I regularly take personal risks when I push managers and executives beyond their comfort level to get to the real problem statement. I often feel like I'm "working without a net" when I pull together a group of stakeholders to perform a blitz-style problem analysis and solution development session, because there's no guarantee of success. And the most common question I get when I'm presenting or teaching is, "How do you tell management/stakeholders/executives to do the right thing without getting your head taken off?"

Generally, I find that the Business Analysis profession provides me with as much risk as I'm willing to grasp. Not everyone need live on the edge, but it's good to be aware that the edge exists and how others navigate it.

Perhaps the "Bad Ass" brand has gotten tired, but the attitude is still valuable in many situations.

...
written by rforsberg, November 12, 2009

Thank you to those who responded.

I do enjoy the content published here - don't get me wrong. Cecile - thank you for the great article. Are the techniques Badassed? I am not sure...

When I think of a badass - I think of someone who ignores the conventional practices and operates in an autonomous and self serving way to get the job done.

I guess I classify someone like Clint Eastwood from Dirty Harry as a badass. You don't assign Harry a murder case, you turn him loose..

Can anyone imagine Dirty Harry as a BA in a requirements validation meeting?

"uhmm.. Harry, we don't see how this functional requirement aligns with the stated stakeholder requirement.." says the project sponsor. Harry pauses as he pulls his smoke from his lips and says, "Do you feel lucky punk?".

Hey, It's just my opinion that a BA is anything but a BadAss according to my definition.

...
written by cecilie.hoffman, November 13, 2009

rforsberg - I agree that under ideal conditions one shouldn't need to have an autonomous, self-serving attitude while doing business analysis. Over the years I learned to be a bad ass in order to get my job done, and, like most techniques, I refined the technique over time, and learned to use it only when needed. My goal here (as a BATimes author) is to articulate the knowledge behind the technique so that others can incorporate the analysis knowledge in their tool kit and not spend years acquiring it. If they don't have to become a bad ass in the process, so much the better. And, if pulling on the bad ass t-shirt (or leather pants) helps them get their job done, then at least they have that option. Can we agree on the value of having that option?

...
written by pmulvey, November 13, 2009

jnunemacher - no problem being the sole BA on a team - ask BAs in other areas to peer review your document. They will have an added advantage in that they are probably unfamiliar with the processes that you are compeltely familiar with, and because of that, they will ask that you clarify your "tribal knowledge". Of course, you may have to trade time reviewing one of their documents as well.

Good point about leaving your emotions at the door. I know that it's good advice, but we all have a personal stake in what we are presenting to others. To have the product of our labors ripped apart doesn't feel good. However, know that the intent is to make the document better for the project and the organization, not to boost your ego. As a peer reviewer, I try and point out what could be changed and leave the emotion out of it. I also point out good items in the document. If the analyst did an excellent job of diagramming a complex technique - I'll call that out. Remember, we can also comment on actions that were done correctly in addition to those that were done incorrectly.

Write comment

You must be logged in to post a comment. Please register if you do not have an account yet.

Business Analyst Times