Skip to main content

Tag: Validation

The BA Practice Lead Handbook 9 – Measuring the Effectiveness of your Business Analysis Practice

In previous articles, we discussed ultimate measures of business analysis success: value to the customer and wealth to the bottom line of your organization. We stated that the real work lies ahead; the work to ensure sustainability, continuous improvement, and delivery of real business benefits. We focused on:

  • Continually increasing the capabilities of your BA team and the maturity of your BA practice,
  • Measuring the business benefits of your BA practice and of projects in terms of value to your customers and/or wealth to the bottom line, and
  • Creating and executing a strategic communication plan to demonstrate the value of business analysis to all stakeholders.

That all seems well and good, but there must be more measures of BA practice success. In article #7, Tom T. posted a comment that asked these important questions about metrics and measurements of business analysis performance, which are essential to the sustainability of your BA Practice (see BA Practice Framework below). The commenter asked:

  1. How can I measure and manage the productivity and quality of my BA team?
  2. Should there be a “standard” timeline for producing requirements? I doubt it, but there should probably be some sort of guideline in estimating how long the process should take.

  3. Are there specific quality measures which can and should be implemented? Probably, but they all “add overhead” to an already cumbersome and time-consuming process.

  4. Without solid historical evidence (of our own), how do I justify the need for additional reviews and inspections?

  5. How do I even get people to accept the need for compiling solid historical evidence?

  6. For me, it’s not about the requirements tools, it’s about the oversight. Tools and templates for that don’t seem to be nearly as abundant.

The BA Practice Framework

hass july16

Sustainability. Demonstrate value through performance measures

So let’s examine the questions posed by Tom T. His first question: ‘How can I measure and manage the productivity and quality of my BA team?’ is really two different questions, one about productivity of the team and the other about quality of the business analysis products and services.


According to, productivity is a measure of the efficiency of a person, machine, factory, system, etc., in converting inputs into useful outputs. Productivity is computed by dividing average output per period by the total costs incurred or resources (capital, energy, material, personnel) consumed in that period. Productivity is a critical determinant of cost efficiency.

According to productivity is:

  • The quality of being productive or having the power to produce
  • In economics, productivity is the ratio of the quantity and quality of units produced to the labor per unit of time

So, productivity measures the cost to construct or manufacture outputs in a production environment when the same product or service is delivered repeatedly. I submit that this measure does not apply to business analysis deliverables. Each business analysis output is unique and tailored to the needs of projects. The question is not: ‘How many business analysis products can our BAs produce in a week?’ Indeed, in this world of scarce resources and time to market demands, the fewer and lighter the BA artifact the better.

Having said this, you as BA Practice Lead should begin to capture historical data on how long it takes to develop, manage changes, and validate requirements for projects of varying complexity. Refer to prior articles for information about the four levels of project complexity. Historical data will enable you to plan new efforts more accurately, and determine what types of projects require longer timeframes.


Quality, however, is another matter altogether. The quality of business analysis products and services is paramount to establishing and sustaining a successful BA Practice. According to, quality is:

  • A peculiar and essential characteristic
  • An inherent feature
  • Capacity
  • A degree of excellence
  • Superiority in kind
  • A distinguishing attribute

Therefore, the essential characteristics of high quality requirements artifacts must be identified and assessed to determine the level of quality. To do so, the BA Practice Lead needs to establish a quality assurance program for the systematic monitoring and evaluation of the various aspects of business analysis products and services to ensure that standards of quality are being met.

Characteristics of High Quality Requirements

Are there specific quality measures which can and should be implemented? Some say for requirements to be useable, they must consist a set of characteristics and attributes. It may be helpful to use a checklist similar to the one presented here to help quickly validate the quality of a set of requirements. Note: the checklist should be different for projects of low, moderate, and high complexity – obviously, the higher the complexity, the more rigorous the validation sessions. See former articles to determine the complexity level.

The characteristics of good requirements can also vary based on the specific business and technology domain being addressed. The following checklist provides the characteristics that are generally acknowledged.

Requirement Validation Checklist


  • Technical team members:
  • Business team members:
  • Author/BA:
  • Others:
  • Original initiating documents to be used to ensure completeness of requirements:
         o Business Case
         o Project Charter
         o Statement of Work


ID Number Quality Characteristic Explanation Criteria Met (Y/N) If No, Defect and Rework Required Defined Another Review Needed (Y/N)
  Clear Requirement is clear and concise so it can be used by virtually everyone in the project. Selected types of requirements are expressed formally using technical language, e.g., legal, safety, and security requirement, and they are mapped back to the requirements that are more easily understood. However, in most cases the language used to document requirements is as non-technical as possible.      
  Unambiguous The requirement is concisely stated without recourse to technical jargon, acronyms (unless defined elsewhere in the Requirements document), or other esoteric verbiage. It expresses objective facts, not subjective opinions. It is subject to one and only one interpretation. Vague subjects, adjectives, prepositions, verbs and subjective phrases are avoided. Negative statements and compound statements are avoided.      
  Visible A diagram can express structure and relationships more clearly than text, whereas for precise definition of concepts, clearly articulated language is superior to diagrams. Therefore, both textual and graphical representations are essential for a complete set of requirements. Transforming graphical requirements into textual form can make them more understandable to non-technical members of the team.      
  Unique The requirement addresses one and only one thing. Each requirement is unique, describing an exclusive need to be met by the solution. Each requirement has an identifier that does not change. The reference is not to be reused if the requirement is moved, changed, or deleted.      
  Complete The requirement is fully stated in one place with no missing information.      
  Consistent The requirement does not contradict any other requirement and is fully consistent with all authoritative external documentation.      
  Traceable The requirement meets all or part of a business need as stated by stakeholders and authoritatively documented.      
  Current The requirement has not been made obsolete by the passage of time.      
  Verifiable The implementation of the requirement can be determined through basic possible methods: inspection, demonstration, test (instrumented) or analysis (to include validated modeling & simulation). Acceptance criteria describe the nature of the test that would demonstrate to customers, end users, and stakeholders that the requirement has been met. Acceptance criteria are usually captured from the end users by asking the question, “What kind of assessment would satisfy you that this requirement has been met?”        

Attributes of High Quality Requirements


Attributes are used for a variety of purposes including explanation, selection, filtering, validating and assuring quality. Attributes allow the BA team to associate information with individual or related groups of requirements, and often facilitate the requirements analysis process by filtering and sorting. Assessing these attributes during quality reviews is one way to focus on the quality of business analysis requirements artifacts. BA Practice Leads should build a checklist of the attributes that are most important to their organization to build in the desired quality of BA products. The checklist may look something like this.

ID Number Quality Attributes Explanation Criteria Met (Y/N) If No, Correction / Refinement Needed Another Review Needed (Y/N)
  Complexity Indicator Complexity indicates how difficult the requirement will be to implement. Highly complex requirements have numerous interdependencies and interrelationships with other requirements. Complex requirements necessitate more rigor to define and model, and more verification to demonstrate their validity.      
  Owner Ownership specifies the individual or group that needs the requirement. The absence of ownership indicates the requirement may not be valid.      
  Performance Performance addresses how the requirement must be met; how fast the process must be executed.      
  Priority Priority of the requirement rates its relative importance based on business value. Low priority requirements typically have a low return on investment, and are likely not produced.      
  Source Source of the requirement identifies who requested it. Every requirement should originate from a source that has the authority to specify requirements.      
  Stability Stability is used to indicate how mature the requirement is. This is used to determine whether the requirement is firm enough to prioritize it and to begin work on it.      
  Status Status of the requirement denotes whether it is proposed, accepted, verified with the users, or implemented.      
  Urgency Urgency refers to how soon the requirement is needed to meet the business objectives, the market window.      
  Functionality The requirement can be implemented so that it performs the functions and features needed, and complies with the relevant standards. Issues related to data security are considered.      
  Usability The requirement is intuitive, easy to understand and to use. The customers/users must be able to perform their tasks in a consistent and efficient manner. The solution appears to be simple, and hides the complex technology from the customers/users.      
  Reliability The requirements have a high probability of failure-free operation of a business process in a specified environment for a specified time.      
  Efficiency and Performance The requirement can be implemented efficiently in the target environment, performing the tasks in an appropriate time frame while utilizing a reasonable amount of resources.      
  Maintainability The requirement is easy to maintain, enhance, and refine as the business need changes.      

Characteristics of High Quality Requirements Development Processes

Another measure of quality is the process used to develop the requirements. If you don’t have a defined process, it is important for you to implement one for your BA team. The process should include the minimal types and numbers of requirements artifacts for projects of differing complexity. Remember, ‘just enough’ process is good enough. Perhaps your process should include the following steps. Again, it may be helpful to use a checklist during validation sessions to ensure your standard process is followed. A set of attributes are presented below for your consideration.

  • Planning requirements activities. Planning the number and type of elicitation sessions to be conducted and requirements artifacts to be produced. Plans should follow your organizational defined standard process. So, Should there be a “standard” timeline for producing requirements? If you plan well, and begin to capture actual data on how long it takes to develop, manage changes, and validate requirements for projects of varying complexity, you will have an idea of the timeline that is appropriate. How do I even get people to accept the need for compiling solid historical evidence? You simply need to require your BAs to develop plans for their activities, capture actuals, and begin to build historical data and a target timeline for projects of varying complexity. It is your job.
  • Studying requirements feasibility to determine if the requirement is viable technically, operationally, and economically
  • Trading off requirements to determine the most feasible requirement alternatives
  • Assessing requirements feasibility by analyzing requirement risks and constraints and modifying requirements to mitigate identified risks. The goal is to reduce requirement risks through early validation prototyping techniques
  • Modeling requirements to restate and clarify them. Modeling is accomplished at the appropriate usage, process, or detailed structural level
  • Deriving additional requirements as more is learned about the business need
  • Prioritizing requirements to reflect the fact that not all requirements are of equal value to the business. Prioritization may be delineated in terms of critical, high, average, and low priority. Prioritization is essential to determine the level of effort, budget, and time required to provide the highest priority functionality first. Then, perhaps, lower priority needs can be addressed in a later release of the system.

Characteristics of High Quality Requirements Validation Processes

Without solid historical evidence (of our own), how do I justify the need for additional reviews and inspections? Requirements validation is the process of evaluating requirement documents and models to determine whether they satisfy the business needs and are complete enough that the technical team can commence work to finalize solution design and begin development. Validation is not an option. It is essential to catch any omissions, errors or defects before further investment is made to convert the requirements into working processes and systems. Build a fast and efficient requirements validation process into your standard BA processes for the systematic monitoring and evaluation of business analysis products and services to ensure that standards of quality are being met. Justify it by assuring that defects found early are vastly less expensive to fix than those found in the test phase, or in production. IBM Systems Science Institute estimates that it is 100X more costly to fix a defect after deployment of the solution than in the design phase, and 15X more expensive in the testing phase.

When using the checklists, the validation team compares the set of requirements to the original initiating documents (business case, project charter, or statement of work) to ensure completeness. Include both business and technical representatives in the review process.

  • Business representatives focus on the clarity and accuracy of the requirements, and
  • Technical representatives focus on whether the requirements are sufficient to finalize design and begin construction of the business solution.

Beyond establishing completeness, validation activities include evaluating requirements to ensure that design risks associated with the requirements are minimized before further investment is made in solution development. An often-used analysis technique to help validate and understand requirements is prototyping to make the proposed solution to the requirements visible. Bring developers into the requirements process; use them to build validation prototypes during the requirements elicitation and validation process.

A word about validation and reviews: These sessions are designed to catch errors, omissions, and defects. We want to catch and eliminate any defects prior to moving into the development process. Catching and eliminating defects early is a good thing. It is also a learning process for all participants, contributing to continually improving the requirements development process and outputs.

Putting it all Together

So what does this mean for the Business Analyst?

For the individual BA, consider these strategies:

  1. In the absence of organizational requirements validation checklists, build and use your own
  2. Work with your PM to plan BA activities and deliverables, and capture actual time and resources used
  3. Work with your BA Practice lead and BA team to improve the quality of your BA deliverables.

So what does this mean for the BA Practice Lead?

For the BA Practice Lead, consider these strategies:

  1. Develop organizational requirements validation checklists for low complexity, moderately complex and highly complex projects and programs
  2. Develop your standard BA practices for low complexity, moderately complex and highly complex projects and programs. Conduct quality assurance activities to ensure all BAs follow the standards
  3. Begin to collect basic data on the time, resources, and quality of your BA Practice using the completed checklists and BA plans.

Don’t forget to leave your comments below.

Project and Solution Scope – The Importance of Both!

Some big news in my life, I got married in July!   In the process of getting married and planning a wedding I found the analogy of the wedding process similar to project and solution scope.

Project Scope and Solution scope – How to explain the difference?
I will get to the PMI and IIBA definitions in a moment, but first I want to throw an analogy out there for everyone to react to.

For a little conceptual fun . . .
Engagement is to Project like Marriage is to Product/Solution

The wedding is the “go live” date and implementation of the marriage, the engagement is the temporary endeavor to create the marriage, the marriage is what lives on.

The marriage proposal is the project charter, likely talked and thought about for months, and finally funded with an engagement ring or other cultural token.  A commitment made by key stakeholders to create a marriage.
The planning begins . . . a budget is created for the wedding, negotiation between the couple and parents on budget, timeline, and scope of the wedding.  Meanwhile the couple begins planning their marriage, requirements of their lives together and visions of what the future will hold.  They work on planning the details of their finances, a home, children and family relationships, spirituality, lifestyle, etc. . . . Most discussed at a conceptual level before, but now they look at the details, it is reality now.   All features of a life together, some need to be ready for the wedding with high priority and some more conceptual and will evolve as time goes on.

A plan is created with all of the tasks needed, timelines, who is involved and budget.  Some parts of the plan are task driven and others are key activities to create the marriage.

A wedding coordinator is hired to manage the plan and make sure everything gets done.

Family gatherings are abundant to figure out design of the wedding, and in all of the heated discussion it is easy to lose sight that it is more important to plan the marriage than the wedding . . . so much focus on one day and yet we need to be focused on what lives on after that day.  The bride and groom try to find time among the chaos to discuss the details of their life together.

An hour-by-hour schedule is created for the wedding day and everyone has his or her part to play, including the rehearsal.

And . . . then there are the vows, the go-live!!!!

What lives on forever is the marriage . . . and just like a project, if requirements change, are missed, or the stakeholders, needs, and context changes the marriage is at risk.  The marriage must be adaptable and prove value to those impacted to be considered successful.

The Bride and Groom need to be BAs in their own right and ensure that they are thinking about the future and an enduring life together vs. being all consumed by the wedding.

The scope of the engagement and wedding is all of the tasks leading up to the wedding day to make the wedding happen.  I compare this to Project Scope.

The scope of the marriage is what lives on and the qualities, features, and capabilities that make it successful.  I compare this to Solution/Product scope.

Project Scope “The work that needs to be accomplished to deliver a product, service, or result with the specified features and functions.”
PMI PMBOK 4th Edition

Product Scope “The features and functions that characterize a product, service, or result.”
PMI PMBOK 4th Edition

Solution Scope “The set of capabilities a solution must deliver in order to meet a business need.”

Some practicality of this for your projects:
– Documenting both the project scope and the product/solution scope is critical to the success and business value of the project. 
– PMs are typically responsible for the project scope
– BAs are typically responsible for the product/solution scope
– It is critical to success for BAs to define solution scope, even if it is not defined or well defined when you join the project.
– Solution/Product scope statements are the features and capabilities of the solution/product that live on after go-live; they are not the tasks that need to be done to deliver the solution.
– Project scope alone (task driven) is dangerous to the project success as there is no common alignment and documentation on what should live on once it is implemented.
– Use a Scope/Context Diagram to accompany a Solution/Project scope statement.  The IIBA newsletter had a great 5 part series on context modeling by By Meilir Page-Jones.  The series started in the Feb 2011 Newsletter. 

Happy solution scoping and happy marriage making!

Don’t forget to leave your comments below.

Dealing with the ‘How’ When You are Looking for the ‘What’

BA_Mar_20_How_2977510_XSIt is the classic problem facing business analysts and other project team members who are collecting project requirements – when asking “what needs to be done?” the responses from stakeholders invariably describe ‘the how’. Many requirements collectors overlook the value of this input, as it can be used as a springboard to more complete, valid project requirements. Here is how you can take this ‘how’ input and extract the most value from it as you create your requirements documentation.

Collect and Separate

Rather than reject the input that describes how to accomplish something, versus getting details on what needs to be accomplished, capture the input as valid and characterize it. As you listen to your stakeholder and record their input, use a flipchart with two columns:  one marked ‘What’ and the other marked ‘How’. Recording instances where the stakeholder has shared ‘the how’ allows you to recognize their input as valid, and pursue the requirement further with questions such as “What would you do with this tool if it was available to you today?” Through this questioning, you can ascertain ‘the what’ behind ‘the how’. According to Mindavation staff member and business analysis expert Haydn Thomas, “Sometimes ‘the how’ reflects the essence of the way your stakeholder understands their requirement. Capturing it, and asking further details, not only validates the stakeholder’s need, but it gives you additional understanding as well.”

 Utilizing this technique also serves to reframe the thinking of your stakeholders, which can yield additional requirements. As the stakeholders see the project team member sorting the requirements into the ‘what’ and ‘how’ it is likely the stakeholders thinking will mirror this approach. Lastly, when repeated, the ‘what’ and ‘how’ separation technique will help instill better habits amongst stakeholders as they will see the focus on obtaining and understanding the ‘what’, and will display a greater tendency to present requirements with a focus on what they are trying to accomplish.

What is missing?

Projects by definition create change in an organization. From the stakeholder’s viewpoint this change could a) fulfill a gap, b) add capability and/or c) increase their productivity or accuracy with the work they perform. If this is true, then something is missing in the existing processes and systems that could enhance their job. A good requirements gatherer will utilize a sense of curiosity to find out what that missing capability is, and what will be required to provide that capability. In addition, characteristics of the missing capability can be explored with the stakeholder to ensure the requirement is fulfilled in a pragmatic, easily acceptable manner. This can be done while still focusing on the ‘what’ – and separating the ‘how’ – should it surface.  In instances where an existing system or process is being altered, stating ‘the how’ can be the best way to understand and capture the requirement into a ‘what’ format. This alteration usually reflects the need of the stakeholder, and describes how a new capability can be incorporated into a current environment without requiring substantial new tools or processes. 

What’s Good About It?

Often stakeholders will share ‘the how’ by actually asking for a process or tool with which they are familiar. This is human nature at work; often stakeholders view this as ‘the what’ when they are actually conveying ‘the how’. They do this because they know of no other example or means of conveying their requirement. If a tool is not mentioned by name, ask for specifics so the tool can be investigated. From there, move on to probing questions about what the stakeholder liked about the tool or process, what they didn’t like about it, and how it changed (or enhanced) the way they completed their work. From this, the requirements gatherer can extract ‘the what’ that is needed to complete project requirements documentation.

Take the Deep Dive

The process of requirements gathering starts with a collection of organizational needs. Ultimately however, detailed requirements including functional requirements, process steps, business rules and performance criteria need to be captured and documented. Although many business analysts are not accustomed to ‘flip flopping’ between high level and detailed requirements gathering, this approach can be effective. With stakeholders that have a definitive ‘how’ in mind, a discussion involving detailed requirements is another way of extracting the ‘what’. This approach can also develop an understanding of the way in which the stakeholder prefers (or needs) to work.

Questions that recognize the tool and process the stakeholder is promoting, yet lead to viable detailed requirements gathering include:

  • When would you use the tool or function?
  • Are there instances when the tool or process you are using had to be augmented by a manual process? If so, when and what was the manual process?
  • In a perfect scenario, how would the tool work?
  • What degree of authority do you have when you use the tool? Could that degree of authority be varied? In what instances would your authority change?
  • What decisions do you make yourself, what decisions are made by the tool or process, and should the decision making approach vary? If so, how?
  • What are the exceptions to the process that require approvals or variances to the normal process?

Check the Viability of the Tool or Process

Many inexperienced business analysts either a) accept the ‘how’ as the requirement or b) ignore the information and fail to recognize the value in the ‘how’ input they receive. It is important to keep in mind that, in the mind’s eye of the stakeholder, the ‘tool’ they are proposing is their understanding of a requirement. Proper, balanced and open minded investigation on the part of the analyst is appropriate to understand the stakeholder’s requirement. A great deal can be learned from this investigation, including:

  • Verifying if there is a tool or similar process that is already available within the organization
  • Discovering functionality that might be beneficial with other stakeholders or on other projects
  • Developing a greater understanding of the process(es) the stakeholder is expecting to utilize
  • Understanding if cross functional relationships between processes or tools exist that might increase organizational efficiency

Care should be taken when investigating these tools or processes however. Common items that should be noted and probably avoided when moving to the solution stage of the project lifecycle are:

  • Stakeholder enthusiasm due to familiarity with a tool. Other tools might be as good or better and not known to the stakeholder
  • Influence from marketing. Effective marketing of a product, tool or service can make it appear there are few or no other choices in the marketplace. Careful scrutiny of these stakeholder requests should be applied
  • Holding on to the current state. Many ‘requirements’ come from an unexpressed desire to minimize change in the stakeholders approach to work. This is not always prudent.

Requirements gathering is a fine art. Successful requirements gathering is best performed by taking full advantage of all the input received from stakeholders. Sorting the ‘how’ from the ‘what’ and investigating appropriately is an important means of validating the expressed requirements, yet staying true to the business analysis objectives.

Don’t forget to leave your comments below.

Bob McGannon is a Founder and Principal of MINDAVATION; a company providing program and project management training and consulting, leadership workshops, keynotes and project coaching worldwide. 

The Cost of Validation

Initial Analysis $100,000
Development and Execution $500,000
Validation Priceless


It is amazing, a client pays 100K to 2M or more to develop a system and $0 to make sure the system works. How do we ensure the system meets our needs?

Solution Assessment and Validation

Solution Assessment and Validation is one of six knowledge areas defined in the BABOK (Business Analysis Book of Knowledge). I recently helped a government client document and execute validation under FDA 21 CFR Part 11 rules.

FDA 21 CFR Part 11 Computer Systems Validation outlines validation methods to ensure the system works as expected and meets the business need. All systems must have a Validation Plan, User Requirements, Functional Requirements, and Test Scripts.

User Requirements are general requirements concerning the use of a system.

Functional Requirements break down User Requirements into how users will specifically interact with the chosen system.

Test Scripts are written for each Functional Requirement.

Test scripts are executed, witnessed, and counter-signed. After execution the organization has a fully documented and auditable trail to show the system was properly installed and qualified to meet the business need and technical requirements.

In addition, a validated client documents Standard Operating Procedures for use of the system. This step forces the organization to take a close look at how the system will be used and why it is being implemented. Any time people spend to think through how and why they use a system is a good thing!


The downside is less flexibility in making system changes. A COTS system that is highly configurable and easily changed is now subject to a multi-level change control process. A simple configuration change may take days or weeks to propose, authorize, document, and implement.


21 CFR Part 11 Validation is worthwhile where the cost of deviation from process greatly exceeds the cost of change. In the case of manufacturing pharmaceutical drugs or vaccines, minor deviations in process can impact the lives of millions of people with drastic consequences. It’s in these very cases that validation is indeed worthwhile to ensure systems meet the safety and security concerns of all.

Jonathan Malkin is a Business Analyst at Plateau Systems. Jonathan provides configuration, integration, documentation, and deployment support services for a leader in Talent Management Systems. Jonathan’s areas of support include 21 CFR Part 11 Validation and customizations to COTS software for which he has won multiple awards. His experience includes work in the federal government, telecommunications, mortgage and banking, and custom software development industries. Plateau Systems is a leading global provider of adaptable, unified web-based talent management software, content and services to onboard, develop, manage and reward talent.