The Tyranny of Best Practice and Its Effect on Requirements Elicitation

tyranny1How effective is today’s requirements elicitation? What do the statistics say? How long is your change request log? Did you ever wonder why there are so many changes during requirements definition? And why can so many project failures be traced back to the requirements?

I’d like to apologize because it’s partly my fault. I don’t mean just myself personally, but rather the generation of business analysts who crafted the foundation for today’s requirements elicitation best practices. That said, the blame shouldn’t be placed entirely on the shoulders of my generation. You see, part of the problem is our inability to learn, organizationally. Organizations have adopted a best practice approach to learning. And therein lies the problem. Let’s examine the serious drawbacks of this learning approach.

The best practice approach is based on evolution. It is slow, agonizingly slow. I don’t know what nature’s purpose is, but I can say with some certainty that in nature it is the strong and the prolific (great in numbers) that survive. Nature is not necessarily interested in keeping the “best”. So it is with best practices. How do best practices emerge?

It all starts with personal experience. We all have it, and we all develop personal practices around it. At some point, an organization may decide that it doesn’t want that many personal practices and it prefers a single organizational practice. So it sets about establishing one. A team is assembled, and based on their past experience, usually 5 – 10 years, they develop an organizational practice. But the one that ends up the winner, isn’t necessarily the best, but rather the one that gets consensus. Often this is a compromise close to the majority practice. Of course, we all know that the best is rarely in the majority. Practices are rarely subjected to verification. And so the strong survive, and we get an organizational practice. Now this practice needs to be propagated to the organization. The time-frame for this can easily be 5 – 10 years or even longer. So at the end of the day we have a practice based on experience that is 5 – 10 years old and which takes 5 – 10 years to spread through the organization. So we have a timeline of 10 – 20 years or more to develop an organizational practice.

Then along come consultants who notice these practices. In some they see an opportunity for business. And so, they get behind these practices. Of course, of the many practices out there, the ones with the greatest opportunity for them, are the ones that will get the most support. And so, these practices start to show up at conferences, in articles and in books. As more and more organizations are convinced to try them, we reach a critical point where hundred of organizations have tried the emerging “best practice”. Among the hundreds, we search for three to five organizations that are both well known, respected and successful. These become the poster boys for the marketing campaign to follow. Their success serves as the fuel for pushing the practice forward. The fact that failure rates for the practice exceed the 90% mark, are never revealed. All we need to show is that it can work, and that it has worked. Eventually, after 10 to 20 years or more, enough organizations are convinced, so we get a dramatic increase in adoption. Now it’s out of the hands of the proponents, the pushers, and into the hands of the adopters. The adopters now begin to talk to each other and stories of failures begin to increase. The true effectiveness begins to emerge. And within five to seven years, another best practice hits the dust.

And how long does all this take? It takes 10 to 50 years to discover that a best practice may not be that good. Can you afford to wait that long to try a practice developed by people based on experiences from decades ago? Six Sigma was branded in 1986 by Motorola based on principles that date back to 1920. It is just now beginning to achieve the level of adoption that immediately precedes abandonment. Lean was primarily developed following World War II by Toyota. The majority of Lean principles taught today were developed between 1945 and 1965. That’s practically ancient history. Again, it is just now beginning to achieve the level of adoption that immediately precedes abandonment. Now abandonment doesn’t mean it’ll completely disappear.

Hindsight is 20/20 – Until You Turn a Corner

Best practices are based on hindsight. The best practices for your job were developed primarily by people of my generation, based on our experience with Requirements Elicitation and Project Management. Unfortunately, our experiences were based on a completely different reality than today’s. We have turned a corner. Didn’t you get the email?

Today’s reality is considerably different. Our job was to automate well defined clerical jobs. Yours is to improve process performance. We dealt with departments with single points of authority. You deal with cross-functional groups with functional biases. We didn’t have change control. Change control has been introduced to fill an apparent oversight on our part. But actually, there was no oversight. We just didn’t have that many changes – our requirements were stable. Today the opposite is true. In fact you can view the length of your change control log as an indicator of risk. The longer your list, the more risky are your requirements.

I appreciate the compliment you are paying us, by basing your approach on ours. But you really shouldn’t have. We were working on cars, and you’re working on jets. Our world moved slowly so we could rely on personal practice to develop best practices. The timelines fit. But at today’s speeds, this approach doesn’t work. We were the more educated within the organization. We worked with dedicated but less educated employees, performing simpler jobs and were willing to be told what to do. You are working with a workforce that is more educated than yesterday’s most educated. Their work is more complex and more varied. And it changes more often and more quickly. As a business analyst, or process analyst, you have little hope of being able to understand to any level of detail all the work required in a complex process simply through interviewing, workshops and software.

But let’s review the conditions prevalent at the time:

  1. Computers were new and expensive.
  2. We didn’t have users yet. We had job experts.
  3. The job experts knew their jobs.
  4. The job experts didn’t know what computers could do. So they didn’t know what they wanted and we didn’t ask them what they wanted.
  5. Departments were full of people who did the same repetitive work and had been doing so for a long time.

So how did we gather requirements? We asked people what they did. We usually asked a few people knowing that everyone didn’t do things exactly the same way. Then based on what we understood from what they did, we designed a computer system to automate their job.

There are a few really important things to understand:

  1. We didn’t get many changes, because there was no opportunity for changes. If you ask someone what they do, and they have been doing it for years, they are not going to come back two weeks later and change their mind about what they did. We asked the job experts a question to which they had a firm, correct, and unchanging answer.
  2. The job experts didn’t have any notions of what computers could do, and so they didn’t interfere in the design process.
  3. The power of the computer to automate the simple jobs was so great, that it was difficult to fail.

How does that differ from today:

  1. You don’t ask subject matter experts what they do. You ask them what they need? The first is a question to which they have the answer. The second is a question to which they do NOT have the answer. The result is that the answer keeps changing. Hence, the long list of changes requests. Answering the question becomes a learning process.
  2. You aren’t in control of the design process. In fact, for most organizations, the design process has been abdicated to the SME committee. The assumption is that because you know how to assemble a car, you must also know how to design an assembly line. Clearly this is a false assumption. Designing and doing are two different disciplines. The people who design race cars are not the ones driving them, and vice versa.

We designed processes by looking backwards to what worked – best practices. We had the time. We had the stability. You have neither. You have little time and lots of change. You need to drop the best practice approach, because “best practices” are no longer a “best practice”. The world has moved on and so has your job. The paradigm has shifted. But the methods have not. That’s the bad news.

The good news is that the future business analyst job is more sophisticated, more impressive, will produce greater results for which you will be rewarded. You will be appreciated for your talent and your unique abilities. You will be seen as part of the business and not just a necessary appendage. But since your job has moved, so must you. Your new approach must be based on designing forward, rather than looking back. This requires that you move away from best practice and towards engineered design driven by performance. It means you have to move away from waiting for the market to accept a principle and towards testing it yourself to see if it works in your environment. It means that you can no longer speak to subject matter experts from across the table to elicit requirements they don’t have, but rather you must lead them in discovering what the higher level process needs in order to produce the desired value. Subject matter experts are not your customers in this exercise. They are part of a team. You are part of the same team, but you have a different role. So do they.

Designing to performance is a totally different paradigm from designing from experience. Designing to performance requires the introduction of science to the art of design. It requires concepts to be tested in real time, rather than distilled through generations of experience. Designing to performance takes the profession to a true profession based on scientific based knowledge, not committee-based knowledge. When medicine was based on best practice, leaches and blood-letting were accepted practices for centuries. Today’s medicine, based on science, has had more advances in one year than was possible in a century under the best practices approach. Best practices dissuade new approaches. Science kills ineffective approaches.

The choice is simple. You can continue to look behind you, after you have turned the corner. Or you can design to the future, while retaining not past practices, but rather enduring principles. You can continue to struggle along a winding mountain road on foot or switch to an automobile on a highway. For sure, it will take some time and effort to learn to drive, but isn’t it worth the ride?

How do you know when it’s time to shift to a new paradigm? When the conditions that lead to the current paradigm no longer exist, then we know we have turned a corner. The time to develop a new paradigm is just before we turn that corner. Unfortunately, we have turned that corner over two decades ago. Best practices are now holding us back. We must shift paradigms before someone switches us.

Don’t forget to leave your comments below

Angelo Baratta PMP, CMC, President of Performance Innovation is a practitioner and researcher. He has combined 25+ years of practical business experience with over 10,000 hours of research and development to produce ePPM Future Blue DesignTM – the art and science of discovering near perfect requirements. Used with Lean or Six Sigma it can help you to deliver from double to 10 times more value and turn Lean into a living and evolving methodology. Angelo’s experience comes from services, consulting, financial and manufacturing sectors allowing him to apply solutions across industry sectors. He is also a writer, presenter, instructor, consultant and coach. Angelo can be reached through