The Risk of Documentation Risk Tolerance

This is an open semi-serious response to Mark Monteleone’s article “As a Project Sponsor, What Is Your Risk Tolerance?”

We open with paraphrased lines from Hamlet: Speak the risk, I pray you, as I documented it to you, trippingly on the tongue. But if you ignore it, as many of our players do, I would gladly have the developers and groundlings make it up as they go.

Risk is uncertainty. If we are certain about something, there is no risk. We are certain the sun will rise during the day so there is little to no risk that we will have to endure total darkness. On the other hand, there would be a high risk of personal damage if we were to run blindfolded into a traffic-filled street. If we were sure we would get hit by a vehicle, it also would not be a risk but a certainty. However, we know that it is possible to race blindly into a line of fast-moving cars without injury because we see it happen all the time in the movies! (Do not try this at home. These are professional jaywalkers.)
There are few products that can be defined to one hundred per cent certainty. A space shot comes close because we are certain about the force of gravity, the amount of fuel to escape the pull of gravity, the weight of the rocket to carry that fuel, etc. There might be some uncertainties because we may have never done it before, but some laws of physics are immutable and therefore certain.

When it comes to business, uncertainty reigns. We may only be sure of one thing: we have a business problem. Hopefully we can be absolutely certain that we have the problem clearly defined, but many times that is not even a sure thing. The requirements document, in any form (use cases, user stories, backlog items or a requirements document) states the solution to that problem in greater or lesser detail. That solution is then elaborated or detailed into greater specifications until the solution team can implement it. As the solution is elaborated, we become more certain of what we are doing and the applicability of our solution to solve the problem, and therefore we remove risk.

While there is always a risk that we are not solving the right problem or that we have not defined the problem well, as business analysts we can remove a lot of risk by making sure the problem is not only defined but agreed to by the business community.

When we define the proposed solution, we also remove some risk—we have an approach to solving the problem so the risk of the problem lasting beyond a specific date is now reduced.

A signatory of a requirements document must understand that there is risk inherent in the document. Whether they are actually aware of this is another matter. In my experience, many signatories have not read the document and will never read the document. And regardless, they assume the document assures them that the risk a problem poses to the organization has been removed or at least will be. The larger the document is, the more assurance they feel that the risks are minimized and that the problem is going to be solved. This ties in with lower risk tolerance for larger budgeted projects; the expectation is that a larger budget buys a bigger requirements document. Just as the quality of the document is judged by its weight (the heavier the better), risk reduction is likely also measured in the same way. Can you imagine the response of that $10M project sponsor receiving a couple dozen index cards that describe the solution and being asked to approve them? Consider: “As an astronaut, I want to pilot the space ship to the moon and return safely to the earth within the decade so that I can get a ticker tape parade down Fifth Avenue in New York”. (This is not a knock against user stories, but rather a comment on evaluating risk by document, which as Mark suggests is often done).

However, in business requirements, a signature should only mean, “I agree that this proposed solution will solve the problem. It is the best of the possible solutions we can come up with and I authorize you to proceed with this solution”. To assume that the document itself, no matter how well written, can reduce the risk may be overreaching and giving the document too much credit. Even with all the best practices Mark lists, which should be followed by all definers of business requirements, there is still going to be much more than a ten percent risk that the solution will succeed as defined. The English language has over a quarter million words (according to the OED) and almost each one has multiple meanings. A requirements document, no matter how carefully constructed, will be fraught with ambiguity. That alone creates a high level of risk.

The biggest source of risk is uncertainty. Even if we spend twice the time we asked for to create the requirements document (and we are usually given half the time we ask for, or less), the document would still contain uncertainties. As we gather information to remove uncertainties, additional uncertainties will arise. We are subject to the basic truth of life: the uncertainty of time. We simply do not know what will happen tomorrow.

I think that Mark’s risk tolerance graph is a great guideline to remind us that we can reduce uncertainty by performing the best practices Mark has noted. However, I am uncertain about the percentages Mark assigns, and business analysts can argue forever about what best practices are and what they accomplish. It’s always good to try to simplify a very complex process into some hard-and-fast rules. Human beings like that kind of insulation from uncertainty. Unfortunately, claiming that nine out of ten projects that assiduously follow the best practices listed in the article will be successful is a terribly uncertain and therefore a very risky thing to say. In other words, the model Mark presents has its own risk.

I say this ironically, not negatively. Because Mark has brought up the issue (and the risk) that those reading about risk may decide to reduce the uncertainty of their own requirement documents by following the best practices Mark enumerates and by gathering more information. And that reduces risk. The real question may be whether the project sponsor knows that the extra time (and money) spent performing those practices correlates directly to risk reduction. We are uncertain that the project sponsor has such knowledge, and as Hamlet would say, “Ay, there’s the rub!” Or perhaps, there’s the risk!

Don’t forget to leave your comments below.