Getting Business Analysis Measures Correct

My local news station had a story that made me think of how easy it is to get performance measures wrong. The Atlanta Police Department recently announced to police officers that revenue from traffic violations will be used for police officer pay raises. Early backlash from residents and police officers from the announcement is that officers will begin enforcing more traffic violations than they have in the past since more violations means more money for them. Department officials later stated that officers will not change the way they decide to enforce the laws. They just want to incent officers to appear in court more to prevent cases from being dismissed. 

I think the Atlanta Police Department (APD) has the right idea in mind, just the wrong implementation. They want to increase officer pay…good thing. They plan on doing that with a specific revenue stream which officers have control …good thing. The bad thing is they were not clear on what they wanted to measure. By leaving the measure too broad they may not achieve the behavior change they desire. Police officers can easily increase violations knowing many people just pay the fine without going to court. 

If APD feels that if officers showing up in court more often will increase revenue then maybe they should measure court appearances. As an example if an officer shows in court 80% vs. 50% they get a raise. If it was only this simple for those measuring business analysis!

The perfect measures elude BA managers or those trying to measure business analysis. A primary issue around business analysis measurement is that the focus is on measuring business analysts, those performing the role of business analysis. Things that are measured include things like how many requirement defects were found or how long does it take a BA to complete a deliverable. The problem with these measures is that it causes behavior that does not align with project goals. In the case of defects the BA may spend too much time on analysis in the hopes that there are no defects or spend time defending themselves as to why the defect was not their fault. Both of these behaviors do not result in positive impact on projects.

The first thing for you to get your arms around is there is no silver bullet when it comes to BA measurement. Business analysis is not manufacturing. There are too many variables at play and you can’t blindly apply measures done at another company to your team. Even be open to having different measures for different teams.

You need to look at current project challenges and right fit measures to change behaviors to address them. One example is lack of clarity around the reason for the project. One of the measures they are considering is ensuring specific BA activities around whether scoping is completed. These activities are known to help ensure clarity around project goals and objectives.

Let’s talk about requirements defects some more. Defects found alone are not an issue. What is an issue can be when the defects are found. If a defect is found too late it can have a serious impact on the project. So do you just measure defects? This may be too broad like the Atlanta Police Department issue. What you can measure is how requirements are reviewed. Does the BA have peer reviews, when is the developer brought in, how long after a meeting with a stakeholder does it take to get validation on requirements, etc. As a result of measuring review activities you can change the behavior of the team which should lead to fewer defects later in projects.

Another thing to be open about is subjectivity. My colleague Paul, has a great presentation on business analysis measures that covers a number of ideas that you can apply to your team. One of those measures is asking a BAs stakeholder would you want to work with this person again. Paul goes on to give more detail questions like were you engaged in the requirements process and did the BA utilize your time properly. Before you start saying you can’t measure that, personalities will come into play. I challenge you to ask yourself why personalities should not come into play. We work on projects for people and with people. How teams get along and collaborate greatly impacts project success. For more on this you can read one of my past posts, No One wants to Work with a Jerk.

Business analysis is not done for business analysis sake. It is done to help improve the business and is primarily performed as part of a project or initiative. BA measures should focus on changing behaviors to improve those projects.

All the best,

Kupe

Don’t forget to leave your comments below.