Skip to main content

The Impact of Business Requirements on the Success of Technology Projects

Written by Cynthia Low on . Posted in Articles. 1 Comment on The Impact of Business Requirements on the Success of Technology Projects

Findings Review

Business requirements serve as the ultimate blueprint for IT project success. That’s no big surprise. What might surprise many are the contents of a first-of-its-kind Business Analysis Benchmark Report by IAG Consulting. Among other things, the report found that companies with poor requirements definition and management spend on average $2.24 million more per project on their major projects.

The Business Analysis Benchmark Report presents the findings from surveys of over 100 companies and definitive statistics on the importance and impact of business requirements on enterprise success with technology projects. The survey focused on larger companies and looked at development projects in excess of $250,000 where significant new functionality was delivered to the organization. The average project size was $3 million. The study has three major sections:

  • Assessing the Impact of Poor Business Requirements on Companies. Quantifying the cost of poor requirements. 
  • Diagnosing Requirements Failure. A benchmark of the current capability of organizations in doing business requirements and an assessment of the underlying causes of poor quality requirements 
  • Tactics for Tomorrow. Specific steps to make immediate organizational improvement.

In addition to the full text report, these sections have also been published as stand-alone white papers for ease of use. All can be accessed from www.iag.biz. 

The study provides a comprehensive analysis of business requirements quality in the industry and the levers for making effective change. The following issues are addressed in the report: the financial impact of poor quality requirements; the information needed to identify underlying issues critical to success; and, the data necessary to target specific recommendations designed to yield performance improvement.

The report finds two basic scenarios for companies:

a) Scenario 1: Project success is ‘Improbable’. Companies might be successful on projects, – but not by design. Based on the competencies present, these companies are statistically unlikely to have a successful project. 68% of companies fit this scenario.
b) Scenario 2: Project success is ‘Probable’. Companies where success can be expected due to the superior business requirements processes, technologies, and competencies of people in the organization. 32% of companies fit this scenario.

Impact1.gif

Almost everyone understands that requirements are important to project success. The data above demonstrates that, while people understand the issue, they did not take effective action in almost 70% of strategic projects.

Effective Business Requirements are a process – not a deliverable. The findings are very clear in this regard – companies that focus on both the process and the deliverables of requirements are far more successful than those that only focus on the documentation quality. Documentation quality can only assure that investment in a project is not wasted by an outright failure. The quality of the process through which documentation is developed is what creates both successes and economic advantage. To make effective change, companies must rethink their process of business requirements discovery.

The following are a few key findings and data from the study:

  1. Impact2.gifCompanies with poor business analysis capability will have three times as many project failures as successes. 
  2.  68% of companies are more likely to have a marginal project or outright failure than a success due to the way they approach business analysis. In fact, 50% of this group’s projects were “runaways” which had any 2 of: 
    • Taking over 180% of target time to deliver. 
    • Consuming in excess of 160% of estimated budget. 
    • Delivering under 70% of the target required functionality. 
  3. Impact3.gifCompanies pay a premium of as much as 60% on time and budget when they use poor requirements practices on their projects. 
  4. Over 41% of the IT development budget for software, staff and external professional services will be consumed by poor requirements at the average company using average analysts versus the optimal organization. 
  5. The vast majority of projects surveyed did not utilize sufficient business analysis skill to consistently bring projects in on time and budget. The level of competency required is higher than that employed within projects for 70% of the companies surveyed.

Almost 70% of companies surveyed set themselves up for both failure and significantly higher cost in their use of poor requirements practices. It is statistically improbable that companies which use poor requirements practices will consistently bring projects in on time and on budget. Executives should not accept apathy surrounding poor project performance – companies can, and do, achieve over 80% success rates and can bring the majority of strategic projects in on time and on budget through the adoption of superior requirements practices.

Making Organizational Improvement

The survey findings made it clear that there is no single silver bullet for making organizational improvement. CIOs must look at making improvement across all the areas of people, process, and tools used to support processes to gain organizational improvement. Only a systematic change to all areas of people, process and enabling tools yields material improvement. 80% of projects from the companies which had made these broad-based changes had successful projects.

The findings of the Business Analyst Benchmark describe both the poor state of the majority of companies surveyed, and, a path to performance improvement. To assist the executive reader, this report also analyzes the data for alternative actions which could be taken to make improvement to separate myth from actions that create benefit. The top three findings in this area can be used to change business results: 

  • Impact4.gif80% of projects where successful from companies with mature requirements process, technology and competencies. 
  • Auditing three 3 specific characteristics of business requirements documentation and forcing failing projects to redo requirements will eliminate the vast majority of IT development project failures. 
  • Elite requirements elicitation skills can be used to change success probabilities on projects.

These above survey findings and the underlying statistics are described in detail within the report. Overall, the vast majority of companies are poor at both establishing business requirements and delivering on-time, on-budget performance of IT projects. Satisfaction with IT and technology projects wanes significantly as the quality of requirements elicitation drops. This report is both a benchmark of the current state of business analysis capability, and, a roadmap for success.

The Bottom Line

The challenges in making quantum improvement in business analysis capability should not be underestimated. Organizations understand conceptually that requirements are important, but in the majority of cases they do not internalize this understanding and change their behavior as a result. The most successful of companies do not view requirements as a document which either existed or did not at the beginning of a project, they view it as a process of requirements discovery. Only companies that focus on both the process and the deliverables are consistently successful at changing project success rates.

For companies that have made the leap to the use of elite facilitation skills and solid process in requirements discovery there are significant benefits. Not only were these projects rarely unsuccessful, these projects are delivered with far fewer budget overruns and in far less time. The report describes the use of poor requirements process as debilitating. Using this word is perhaps unfair, since companies do not collapse as a result of poor quality analysis. In fact, IT organizations and the stakeholders involved will overcompensate through heroic actions to attempt to deliver solid and satisfactory results. However, ‘debilitating’ is an accurate word to describe the cumulative effect of years of sub-optimal performance in requirements analysis when results are compared to competitors who are optimal. Even leaving out the effect of high failure rates and poorer satisfaction with results, the capital investment in information technology of the companies with poor requirements practices is simply far less efficient than companies that use best requirements practices.

To illustrate this inefficiency of capital expenditure on technology at companies with poor requirements practices, use the average project in this study ($3 million):

  • The companies using best requirements practices will estimate a project at $3 million and better than half the time will spend $3 million on that project. Including all failures, scope creep, and mistakes across the entire portfolio of projects, this group will spend, on average, $3.63 million per project. 
  • The companies using poor requirements practices will estimate a project at $3 million and will be on budget less than 20% of the time. 50% of time, the overrun on the project both in time and budget will be massive. Across the entire portfolio of successes and failures, the companies with poor requirements practices will (on average) pay $5.87 million per project.

The average company in this study using poor requirements practices paid $2.24 million more than the company using best practices.

If overruns are common at your company, or if stakeholders have not been satisfied with more than five of the last ten projects larger strategic projects, there is definitely a problem and your company is likely paying the full poor requirements premium on every project.

Keith Ellis is a Vice President at IAG Consulting, specialists in eliciting and managing business requirements for technology initiatives. Mr. Ellis was co-founder of the elicitation company Digital Mosaic (merged with IAG in 2007) and has extensive experience in technology research, business analysis issues. He regularly publishes articles, white papers and other research findings in these areas. Mr. Ellis can be reached at (905) 842-0123 x228. Email IAG by accessing www.iag.biz and selecting contact us or call our North American Toll Free line: 800-209-3616

Road to the Perfect Project – Part 3

Written by Cynthia Low on . Posted in Articles.

Limit Formal Process

Many projects today, especially the larger, ones are run based on what I call “big process”. Development organizations get their systems and software engineering processes and procedures CMM/CMMI certified. This means that anyone working on their projects lives in a world controlled by a myriad of plans and processes covering all aspects of that project. Projects in these organizations will also be documentation heavy – typically based on IEEE standards. There are very good reasons for the existence of big process. These processes represent best practices distilled from decades of corporate experience. Big process does provide order and consistency to projects. The intention is to ensure consistent results. The project team just follows the system cookbook and all comes out well.

The biggest problem with this approach is that all the process is, at its heart, intended to do is allow average quality staff to reliably deliver “acceptable” systems. And most of the time it delivers. Now, if all you want is an acceptable (read mediocre) system, then by all means go for it. For big process to work properly the project team should come onto the project already immersed in the company’s methodology based on actual prior experience with it. A very real problem with this approach in the government contracting world is that the development contractors do not keep stables of ready and willing people who have been deeply immersed in the company’s methodology on prior projects. When they win a contract they go out and hire mostly new people for the project, who then have to go through a long period of training and adjustment to the company’s methodology.

Big process tends to force big project size – and big project cost. Typically there exist separate sub-organizations for requirements management, systems engineering, software engineering, development, testing, configuration management, quality control, risk management, etc. And all of that process increases the amount of work (much of it administrative in nature), and also increases the communication load on the project. On a project with a smaller team, much can be done via informal channels where formal process would be needed in larger projects. Proper choice of support tools can improve productivity, and those tools come with built-in process. So using them gives you desirable best practice methodology without the need for separate formal written processes to deal with.

The primary uses for project documentation are (a) to allow the system to be developed and fielded, and (b) to support post-implementation system maintenance. I recommend limiting formal documentation to that which is truly needed. I believe a project needs a high quality set of formal system requirements, and a matching system test plan. Some level of design documentation is needed, with the higher level design being more important than the more detailed design. Depending on system complexity as well as security needs, other documents will be necessary. I believe it becomes a waste of money to continue to maintain anything but the highest level documents for purposes of system maintenance. Below a certain level of documentation needed for orientation purposes, maintenance people assume (largely correctly) that the documentation is out of date, and in any event they often prefer to just directly go to the code.

Exhibit Agility

Given that other conditions are extant, namely there is a small project; it is staffed with a solid leadership team; the project team is in a partnership mode with its customer; and the project is not overburdened with formal process, it becomes possible to take the final step to achieving the perfect project. The two most successful projects I have personally been involved in met these criteria, but also exhibited characteristics that now are associated with what is known as the Agile Methodology. The essence of “Agileness” is the ability to let things flow freely, and to operate intuitively versus slavishly following a rigid set of rules. It requires a high tolerance for ambiguity, and a willingness to quickly change direction, perhaps even radically. The team has to be fast on its feet.

Agility can be exhibited in any manner of ways. From the perspective of the project manager, that person may express agility in being able to rapidly shift assignments, based on knowledge of their subordinate team member’s individual experience and capabilities. This might be in response to a technical roadblock, a customer shift in priorities, or loss of a staff member. An individual analyst may need to multiplex multiple assignments, and be willing to instantly shift what they are doing based on project needs of the moment.

Agility also means thinking out of the box about how things are done. Consider system requirements. While I am a big believer in starting out with a solid set of formal requirements, one has to understand that those requirements are just an initial guess at what the system’s users truly need. It is right and proper to put together an early version of the system based on those requirements. But early pre-release user exposure will almost certainly identify various needs for adjustment. There may even be a need for significant rework. The agile team will, quickly and gracefully, make accommodation for them, even if major rework is needed. The users will deeply appreciate it. But at some point, typically after the first system release (but perhaps even earlier), it will make sense to just abandon those original requirements, and any attempt at traceability to them. The agile team then goes into change management mode and allows the system to morph into whatever it wants to become, as user complaints and suggestions for improvement pour in.

Not everyone is capable of what I used to call “go with the flow development”, and not every customer will allow it, but a project team that is able to operate intuitively can produce absolutely spectacular results.

John L. Dean is a seasoned IT professional with 40 years of varied experience, including systems engineering, systems analysis, requirements analysis, systems architecture and design, programming, quality assurance, testing, IV&V, and system acquisition, equally split between technical and management. He has worked both the contractor and government sides of the fence, has been an employee of large (IBM, CSC, Booz-Allen, MITRE, ACS) as well as very small companies, and is currently an independent consultant. He has a Masters degree in Operations Research from Clemson University. John is the “Project Whisperer”. His depth and breadth of experience gives him an unusual perspective on what does and does not work. If your project is running you, he can use his experience and intuition to diagnose why and apply calm, assertive energy to start corrective action, so you can and regain control! He can be contacted at [email protected].

© John L. Dean, 2008

Leaping Ahead in the New Year

Written by Cynthia Low on . Posted in Articles. 1 Comment on Leaping Ahead in the New Year

Feb1_160x113.gifWe’re just into our second month of this Leap Year with our “once in four one day more” 29 day February and already we’re up to our necks in activity, getting each Business Analyst Times online, planning for the next one and organizing a year’s worth of Business Analyst Worlds.

We’ve managed to pull together a pretty good Business Analyst Times for the first week in February. John L. Dean continues his highly interesting and informative series, Road to the Perfect Project. This time out he takes a look at the importance of having a high quality set of senior analysts on any project. He also discusses how the PM and BA team can work effectively with the customer. Jim Swanson identifies the requirements phase as the point where business meets IT in his article, Writing Effective Project Requirements.

Our intrepid bloggers Terry Longo and Marcos Ferrer are back with their distinctive views on different aspects of our business. I’d also like to point you to our poll question from the last issue: we asked: Can the roles of Business Analyst and Project Manager be effectively combined? The results were: sometimes 45.8%; yes 32.2% and no 22.0%. Only 22% of you said categorically “no” while 78% felt they could be combined. Do you agree? 

As usual, I hope you enjoy this issue and please give us some of your ideas for future issues. It’s the kind of input we need to keep our site truly relevant to you, our readers.

All the best.

Adam R. Kahn
Publisher
Ph: 508-309-6900
Email: [email protected]

Enterprise Analysis: Process or Content?

Written by Terry Longo on . Posted in Articles.

In earlier posts, I have been reasoning that the IIBA and BABOK are on their way to defining a generally applicable framework for requirements management best practices, and that the BABOK 2.0 will be the first significant manifestation of the distinction between process (the requirements life cycle) and content (the nature, or domain, of the requirements being managed).

Viewing the requirements management life cycle in that way, however, requires us to scrutinize the way in which Enterprise Analysis (EA), as currently defined in the BABOK, fits in:

  • EA is content because it is carried out specifically in the domain of business planning and business strategy.
  • EA is process as well, because it is a crucial first step in the requirements life cycle.

The question becomes: is there an EA-like step in the requirements life cycle within other domains?  Let’s explore this question by considering another domain, that of enterprise e-learning infrastructure and delivery (there are many other domains we could choose as well).  From the business strategy point of view, this is a tactical and operational aspect of the enterprise.  But from the e-learning director’s point of view, there is, of course, strategy involved, addressing questions about workforce trends, globalization of the enterprise, the presence of electronic performance support practices, learning-related data privacy, and many others.

So the e-learning director must:

  • Create and maintain the e-learning architecture
  • Conduct feasibility studies of e-learning infrastructure and delivery solutions
  • Determine the scope of e-learning infrastructure and delivery projects
  • Authorize the initiation of those projects
  • Interface with the project team to manage the projects’ value, track benefits, manage change and risk, etc.

Hmmmm.  Looks a lot like BABOK’s Chapter 2: Enterprise Analysis.  Which is the point being made. “Enterprise” is the content.  “Analysis” is the process.  An e-learning director reading BABOK Chapter 2, while pretending the title was “E-Learning Infrastructure and Delivery Analysis” would get some great guidance.

Building on Building the Right Thing

Written by Marcos Ferrer on . Posted in Articles. 1 Comment on Building on Building the Right Thing

As a senior BA, I increasingly find the BA Times to be a fantastic resource for winning hearts and minds.  Kent McDonald wrote a terrific article last month about Value and Building the Right Thing.  It is a must read for any BA who is unsure what to do about difficult stakeholders (are there any other kind?).

In thinking back on numerous projects over the years I could suddenly see what Kent described – teams that were most excellent at building things right, but who were relying on stakeholders to identify and prioritize what to build.  This behavior has derailed a lot of projects (death by “superuser”*), and made Scott Adams a LOT of money for “requirements” jokes.

 

The ability to INFLUENCE a project to perform due diligence (BA) on what is possible AND worthy to build (as opposed to “recommend and retreat”) is a very advanced skill, requiring business and technical knowledge, plus people skills of significant depth and breadth.  This does not describe the experience set of most stakeholders, or even of many BAs (fundamentals, fundamentals, fundamentals!). 

 

Add to the above the natural reluctance of most stakeholders to change.  Since projects imply change, it is actually unreasonable to expect stakeholders to perform a thorough, balanced investigation and analysis.  That is our job.  Then we create as much consensus as possible.

 

Enabling this balanced investigation to happen IS a key BA move – it is the reason we are expected to speak ALL the dialects – “business” and “project” and “IT” and “user” and “developer” and “vendor”.  Failure to  find this balance can lead to projects that are infeasible, unbeneficial, too expensive, unacceptable to some or all stakeholders, and in some cases just funny (did you hear the one about the team that decided to re-invent accounting for the stakeholder who had to receive and cut checks across thousands of accounts, but who didn’t like accounting systems?).

 

SOOOO, deep breath.  Back we go to our exercise in identity requirements.  The reason I am walking through this exercise with you, the fearless reader, is that my experience has taught me that my value is focusing on value.  At this time, I can think of no greater value to freedom and democracy than a careful analysis of the requirements for identity systems.

 

BUT, loyal reader, I am out of time this month.  Here is the problem we have posed:

 

FOR NEXT MONTH:

 

To reassure ourselves that we REALLY understand the stakeholders for identity systems (see prior columns), we will try to list the “identity transactions” that might occur in society, and we will try to match these transactions to the kinds of stakeholders we are aware of so far (individuals, businesses, government, and other organizations).

 

How many identity transactions can you think of that have significant differences in identity requirements (purchase goods, fly to Palestine, get a driver’s license, buy a gun, become a citizen, visit Niagara Falls), or how would you elicit such a list?  Is this why no one has ever done it?

 

Potential answers will be discussed next month, and incorporated into the case study.  The best reader response will get acknowledged next month (send a picture with your response!) and will undoubtedly receive a large raise in the near future, just for rising above the pack.

 

 ALSO:  A tip of the hat goes out next month to anyone who shares a story about “death by superuser”.

 

© 2008 Marcos Ferrer