Beware Proxy Measures
Organizations are usually pretty good at measuring and counting things. Whether it’s positive customer reviews, staff engagement, average call handling time or something else… chances are that someone in the organization is tracking it. They might even be creating reports or dashboards so that executives can see how different elements of the organization are performing.
From time-to-time a metric will drift and there will be a desire to get it back on track. Perhaps the staff engagement survey shows that people aren’t happy. Or perhaps there’s a contact center where the average call handling time has drifted from 3 minutes to 5 minutes. Either way, it’s easy to imagine that a concerned manager would want to investigate and initiate changes.
Yet here a danger awaits the unprepared. It would be very easy to make knee-jerk reactions when referring to the data alone, without considering its context. It would be even more dangerous to make decisions based on ‘proxy measures’. By ‘proxy measure’ I mean some kind of indicator or metric which approximates how well something is going, but doesn’t directly measure it.
That might sound abstract, so here’s an example. Like many people, I wear a smartwatch that counts my steps. Doing this has definitely changed my behavior, and I strive to get 10,000 steps each day. Yet if my overall goal is to “stay healthy by staying active” then the step counter is at best a proxy measure. Sure, it’ll indicate if I’ve suddenly slumped into a sedentary lifestyle… but it would be very easy to ‘cheat’ the system. Ten thousand slow steps around the house are probably not anywhere near as beneficial as fast-paced walking (or jogging)… and that’s before we even consider the fact that it’s possible to wave your arms around to get a few extra steps. I’m sure I’m not the only one who has done that to get an extra few ‘steps’ in before midnight…
The Danger Of Unfair Comparisons
The point here is that it would be easy to equate ‘number of steps’ with ‘how active and healthy’ a person is. But that would be a dangerous equivalence to make. Ultimately, the smart watch is (I guess) “measuring the number of arm movements which are likely to indicate steps”. That is the real metric… it can be used to approximate many other things, but that is just an approximation. You certainly wouldn’t want to rely on it for decision making.
A similar pattern exists within organizations where unfair comparisons are made. Let’s imagine a call center manager is measuring the ‘average length of call’, and wants each agent to achieve an average of 3 minutes or less. The manager is probably equating “effectiveness of operator” with “length of call”. But is this truly the case?
Extending this example, perhaps there are two different agents: One (Agent A) has an average call handling length of 5 minutes, the other (Agent B) of 2.5 minutes. Agent A is put on a performance management program, while Agent B is given a bonus. Is this fair? Or could it be that Agent A is thoroughly investigating the customers’ needs, solving root causes so they don’t have to call back again, whereas Agent B is just doing the quickest thing. Perhaps Agent B even cuts an occasional customer off to hit their target…The point here is that without further investigation it would be impossible to know.
Advertisement
A Key Question: “Why?”
As with so many situations, a key question to ask is “why?”. In this case it’s important to ask why particular measurements are being taken. It can be a difficult question for stakeholders to answer, and different stakeholders might have different perspectives on the rationale for measuring and reporting on a particular metric. That’s useful to know too.
Asking this question can help us to determine potential gaps in the way that situations are being assessed. For example, imagine we asked two stakeholders why call handling time was measured. Perhaps they say:
“To measure efficiency of the call center agents”
“To ensure good customer service”
Arguably, the measure on its own doesn’t achieve either of these. It might be that other metrics are necessary alongside this to give a better picture. Customer feedback, customer satisfaction scores and so on might also need to be considered to give a better picture. In some cases it might be useful to stop measuring or reporting on something entirely, as the very act of reporting just acts as a distraction. All of this depends entirely on the context, so further investigation of the situation is likely to be needed.
Questioning The Norm
As with any situation, this is an area where BAs can add value by acting with curiosity. Working backwards to understand why things are measured will help ensure that possible options for improvement are generated. It will often involve questioning the norm, but most BAs are used to that!