Skip to main content

Tag: Project Management

BA Rising

So, last month I covered Business Analysis, and How I Came to Realize What It Was And What It Meant. This month, I want to go more into what it means, and why the increased influence of BA (and the CBAP certification) is so important.

The short version is this: Systems are increasingly important in our quality of life (are you on the no-fly list yet?), and good BA leads to good systems, bad BA leads to Gartner and Standish Group negative statistics (if you don’t know what these groups do, Google them). BAs don’t always have the influence it takes to prevent these bad outcomes, but they should, and they will, now that the IIBA has established some neutral standards.

So, why BA – why can’t we build good systems without it? PMs are trained to deliver on time and under budget, so what’s the problem? (Hint – How hard is it to deliver on time and under budget? How much have you got to spend by when?) If you read the literature, you will see that MOST causes of project failure are related to stakeholders and requirements. These are failures of BA, by definition of the profession, and by default since they are not key to PMI.

I want to make the statement that the techniques of BA work because they are exactly those of due diligence. Sometimes we say that BA is just common sense, then laugh when we realize how uncommon it is. Due diligence IS common sense. Surely no one objects to due diligence? Indeed, why should anyone be concerned about the rise of due diligence? Isn’t that for dullards?

Before anyone spends millions of dollars (or even thousands), or takes any risk of consequence, it pays to do some amount of homework. If the homework seems overwhelming, expensive, or incomprehensible, that just means that the first 10 hours of homework are VERY important. They may be even MORE important than the next 100, or even 1000 hours. There is a LOT of earned value in doing ANY thinking versus none. Once thinking gets to thousands of hours, there is increasing risk of ossification, and the benefits diminish. In spite of this, a common complaint is “We don’t have time for analysis,” as if none were better than too little.

A common project mistake is to assign too much earned value to development, and not enough to analysis. Million dollar mistakes rarely happen at the field definition level, and are typically easy to explain and correct. This is true in spite of the fact that a significant percentage of labor and time may be spent on field level issues, or other technical concerns. Most of the value and risk control in the project may go to the first hours of analysis, with the development being quite low risk, as the “big” problems are actually resolved. It pays to control costs with analysis, and does NOT pay to control it with code change control.

This seems obvious enough, but analysis is overlooked and under-respected on more than a few projects – witness the 35% success rate of projects!t is, in fact, overlooked by the society in general. My father was once a Quality Analyst for a bank. “What do you do?”, I asked? “I make people think.”, he said. “Thinking is painful, and left to themselves, many won’t do it.”

Hmmm. The problem is, that the lax attitude about thinking is due to the illusion that we understand something is so strong. “I’ve been doing this for years, I know what to do!” In effect, EVERYONE thinks they understand, therefore, there is no need for an analyst – no one is (admits to being) confused – that would be awkward, scary even. In my experience, EVERYONE thinks they are THE analyst. It is, apparently, a strong hero role, even if we are only legends in our own minds.

Think about George Washington, fighting a desperate and losing war with England. In the midst of his desperation, he invented enterprise analysis, from a sheer common sense need – “What does the continental army consist of, what is it not, and what can we do about it?” That is bravery, true action, in the face of seemingly insoluble problems.

Imagine making these decisions without a sense of the whole enterprise. Imagine making these decisions under illusions of how coherent the enterprise was, instead of basing them on the best intelligence available.

Imagine pushing resources around “just because you can”.

And if you want to imagine those things, just imagine working on any project for the last 100,000 years, because human nature is such that it prefers illusion to facts, egos to inventories and simplistic principles to problem solving. Our progress comes in spite of ourselves, because nature supports common sense, and punishes lack of care.

The saying that ” common sense is none too common,” applies especially to BA. BA is the refined, professional level, experience based, highly predictive form of common sense. BAs don’t cite common sense; they practice it. They practice it because they are willing to learn, and think, in spite of the petty discomforts.

An example of how rare common sense actually is: Everyone knows you have to think of your customers to have a successful product (at least everyone would select it correctly on a multiple choice test). In spite of this fact, IBM was able to release the PCjr, with no applications that anyone cared about. IBM followed this up by dissing any employees who pointed out the problem. IBM’s slow decline in the late 80s and early 90s was inevitable, and that IBM no longer exists.

Professional BAs really believe in practicing due diligence, because they have seen what happens when you don’t. They are not in denial, and not afraid of the truth. They believe it, and live it, where others don’t, for whatever reasons. YES, they can see the future, because they remember the past.

In a world where “doing it because you can” is the message from the top, it’s easy to abandon due diligence – the paychecks can flow for years sometimes. It is much harder to adopt the Quixotic position – things are worth saving, improving; the world deserves to be a better place.

Given the importance of systems to the state of the future world (identity systems, micro-cash, medical imaging, health management, internet security, etc.), does anyone doubt that they want BA to rise?

This is an important question, for the following critical, political reason: If BA represents stakeholder value, doesn’t that mean that the Project Manager actually reports to the BA, instead of the other way around? Why or why not? Don’t most PMs wish they had a better handle on the requirements before the deadlines and budgets are set? Stay tuned.

Our society has hardly started building systems. We have traffic managements systems to build that will save lives, medical management systems that will empower patients, and identity systems that benefit the users (us) as well as the implementors (them?).

If anyone is against BA, they are against common sense, stakeholder interests, due diligence, and successful outcomes for a democratic society. If you are out there, and don’t want to play, please stand up, so the rest of us can work around you.

Putting the User Back into User Acceptance Testing

Why is getting users involved in User Acceptance Testing (UAT) so challenging? Isn’t it called UAT because the users are the main participants? My experience has shown that involving users in all phases of the project, especially UAT, is the best way to ensure project success.

I recently worked on a project in which a major defect was found after the software application moved to production. This defect caused the users to perform three days of manual processes. Users on the IT project team worked countless overtime hours. The defect also resulted in a frustrated user group and business sponsor. The project team’s morale was low and the business users lost a great deal of confidence in the project team’s ability to deliver quality software solutions. To reduce the risk of making this crucial mistake in the future the project team improved the UAT approach by getting users more involved.

Traditional Approach

Too often, User Acceptance Testing is not taken seriously. For many reasons UAT gets shortened, is not conducted in a way that ensures a successful project or, the worst scenario, is not done at all.

An approach I have used in the past consisted of the project team members—Business Analysts and QA Analysts—writing test scripts and providing demos of the new application to the users. The users would then walk through test scripts step-by-step. In some organizations the BAs write and execute the UAT tests and then present the results to the users for sign-off.

Traditional approaches are often not effective because they are missing a key ingredient—the right amount of user involvement. In the project previously mentioned, there were five major issues relating to UAT that we had to address. These are common problems in many organizations related to lack of user involvement:

• Users may not be fully vested in UAT. In the traditional approach the users are directed by the BA during UAT and are brought in too late in the project to have an impact on the test plans. This results in a lack of ownership by the users and less responsibility on their part for the success or failure of the project.

• Users do not fully understand how the new functions should work when they are asked to test them. Just seeing a demo is typically not enough. This can result in the UAT session becoming a training opportunity and not a true test.

• Tests are often generic and are not all based on real-life scenarios. If the test scripts are written by the IT project team, there is a greater risk for missing real-life scenarios. This is because, unlike the user, the IT project team does not use the application every day.

• Project team members are usually pressed for time. Often a BA has already been assigned to perform requirements activities on another project during UAT. Balancing multiple projects means that BAs have a hard time focusing on UAT, while meeting their other project deadlines.

• High pass rate of UAT test plans. Ironically, this is not a positive thing. Often a BA writes test scripts and tests them himself prior to UAT to ensure the scripts pass. When a BA writes the test scripts the users are not given an opportunity to interject enough real-life scenarios to validate the system.

A Recommended Approach

To address these common issues and increase the chance for project success we need to take a new approach to UAT.

1. Involve key users early

Once Quality Assurance (QA) begins testing, UAT planning should start. Identify users who have a deep understanding of the business requirements and are change agents for the group. Identify all of the tasks that need to be accomplished, the owners of each task, and a high-level timeline. This will help ensure that all the right people are involved.

2. Provide hands-on training of the system for the UAT participants

Providing a demo is not good enough. Once QA feels the application is stable enough, give hands-on training to the UAT participants. It is critical to explain to the users that issues may arise because QA testing is not complete. Ask the users to stay focused on how the application works and not so much on the fact that it is not fully operational.

3. Use facilitation sessions to create test plans

Have the users write their own test plans. This may sound far-fetched, but it is key to getting UAT as close to real life as possible. The BA’s primary role is to facilitate the UAT test plan creation process, but not to write a single test script. Using process workflow diagrams and Use Case documentation from the requirements package, ask the users to determine what processes and system functions need to be tested. Provide the UAT participants with examples of test scripts and explain the need to capture the goal of each test, the necessary steps, and the expected results. The steps become second nature to the users of the system, and they often find it difficult to document each step they take to accomplish a goal. Help them think through their processes in detail to ensure they have documented each task completely.

Review the test plan. Once the test plans are written, the BA reviews the test plans to ensure all the necessary functions and processes impacted will be tested.

Determine necessary inputs and outputs. Once all the test plans are written, ask the users to document the inputs they need to complete each of their test scripts and the outputs that will be generated. Make sure all UAT participants have the necessary inputs to complete their tests based on all of the outputs. If some are missing, enlist other users to create those inputs during testing execution.

Make it as close to real life as possible. To enhance the real life feel, the BAs work with the users to determine a testing schedule. Make sure the schedule follows their daily process. Again, use process workflow and Use Case documentation to ensure the test plans are executed in the order the activities would be done in real life.

4. Ensure users execute the test

The BA’s role is to ensure the test environment is set up and to assist the users as they execute the tests. The user’s role is to execute their tests and document the results.

The Results

Recommended Approach to UAT

1. Involve key users early

2. Provide hands-on system training for the UAT participants

3. Use facilitation sessions to create test plans

4. Ensure users execute the test

Using this approach can help reduce the risk of major defects making it to production and ensures the users are satisfied that the solution meets the objectives of each project. Here are some of the key results from using the improved approach:

• The UAT participants take responsibility for the success of the release. They feel part of the team due to their collaboration with the IT project team and involvement in UAT planning, and test creation and execution. They also help champion the benefits of each release to the larger user base.

• Due to the pre-test training, users are comfortable with new applications. This allows the users to develop real life test scenarios, and the time allotted for testing is not used for training.

• Since the users create and execute test plans, the tests are very close to real life scenarios and the users are more comfortable running the tests.

In addition there are tangible benefits to future projects:

• QA incorporates the scenarios documented by users in the QA test plans for future releases.

• Over time there is decreased use of the BA’s time for UAT. With the BA facilitating the UAT process and not doing most of the work, the BA can focus on other necessary tasks – like launching the next project.

Implementing the Approach

A lack of user involvement in UAT is not uncommon. I urge you to try this approach even if you have not yet experienced a drastic wake-up call, such as major defects in production.

As we are called upon to deliver solutions faster and faster, it is just a matter of time before major defects make it to production. Here are some tips for getting started:

• Start small. To help manage the changes with the new approach, identify a release with a low number of users and/or new functions. This will allow you to test the new process, discover lessons learned, and make the necessary adjustments.

• Plan for additional time. Using this new approach will initially require more time. Work with your Project Manager to plan more time into the UAT phase for your first two or three projects. As you get accustomed to this approach, it will require less time.

• Identify power users and champions of application. They are your best testers and have the most interest in the project’s success.

• Sell the benefits of the new approach to your users. As with any new approach, BAs need to help the users understand what the approach is, and how it will ultimately improve their business.

• Save the user test plans for future releases. Reuse of test plans will help speed up the time dedicated to UAT in future releases and can be used to update your QA test plans. Successful implementation of this approach helps ensure projects meet the user needs. The collaboration of users with the project team leads to a shared responsibility for the success and failure of the project.

What is UAT and Why We Do It?

UAT is the final approval by customers signaling the new system or enhancements can be deployed. UAT is unlike other types of software testing (e.g., unit testing, system testing, integration testing), because during UAT we are looking for conformity. We need to validate that the solution meets the business objectives and works correctly with real-life scenarios. UAT is typically conducted by users with assistance from the BA and other project team members.
UAT is most often conducted before a system is deployed into a production environment. For higher risk projects UAT may continue for a period of time while running the old system and new system in production. This gives the users ample time to become comfortable that the new system meets their needs. For commercial software companies UAT is also know as “Beta” testing. Here the system is launched into a production environment, but only to a subset of customers who will provide feedback on defects and necessary improvements.