The Unbiased Analyst
Whether it be interviews, surveys or questionnaires, business cases, observations, or root cause analysis – any technique can introduce bias into a business analysis initiative.
Bias can result in the collection and synthesis of erroneous information leading to the delivery of sub-optimal, discriminatory, and/or inadequate solutions. It is therefore important that Business Analysts are aware of the different types of bias and the risk they pose to any analysis.
The following outlines some common forms of bias, as well as provides some simple approaches for mitigating the risk of introducing bias into an analysis.
In cases where there is a large group of stakeholders, it is often not practical to elicit information from every individual. Instead, information is elicited from a selection of group members.
Selection bias is where individuals are more or less likely to be selected from the larger group. The risk posed by selection bias is that the information solicited from the selected individuals is not representative of the larger group.
Any business analysis technique that involves selecting individuals to represent an entire group is at risk of introducing selection bias. For example:
- Surveys and questionnaires often depend on individuals taking the time to respond, which may be depended on a range of variables, including being aware of the survey/questionnaire, having access to complete the survey/questionnaire, having the time to respond, and/or having the ability to respond.
- Interviews usually involve selecting individuals or asking for volunteers from a larger group.
- Observations may introduce selection bias based on when and/or where the observation takes place.
It is also important to be aware that any elicitation that involves self-selecting is inherently biased towards those who chose to be involved.
Response bias is a general term that accounts for range of bias that influence the response of participants away from an accurate or truthful answer. Some areas that may contribute to response bias include:
- Recall – human memory can be unreliable, particularly if the information being recalled is complex, regards an event that took place a long time ago, or involves information that may be considered traumatic.
- Perception – this is where a response is influenced by what is perceived as a ‘good’ or ’bad’ answer. The responder essentially changes their response or behavior based on what they think the interviewer/observer wants to hear/see, ultimately leading to the collection of erroneous information. An individual’s response can also be impacted by how a question is framed. For example, using overly negative language, positive language, or language that is associated with a particular ‘side’ can influence response.
Any business analysis technique that involves collecting information direct form stakeholders may be at risk of introducing response bias.
Personal bias is where an individual unintentionally or unwittingly introduces unwarranted opinions and feelings into a situation, making objective opinions difficult. Business Analysts are at risk of introducing bias into an analysis because of their own views and believes.
Some particular areas of personal bias include:
- Observer Bias – this is bias introduced when an individual’s point of view or understanding impacts how they perceive a situation. For example, an observer may misinterpret a situation because the language used has a specific meaning in the given context that the observer is not aware of.
- Confirmation Bias – this is the tendency for an individual to seek out evidence that confirms their own personal views, while downplaying or discounting evidence to the contrary.
- Overconfidence – this is where the credibility and/or knowledge of an individual or source is overestimated.
Introducing personal bias into analysis is a risk for any Business Analyst regardless of the technique being used.
The following is a non-exhaustive list of ways Business Analysts can mitigate, counteract or avoid introducing bias into analysis:
- Understand your stakeholders – take the time to understand your stakeholder groups, including understanding attributes that may impact the particular needs of individuals within a group. Attributes to consider include demographic information such as age, geographic location, socio-economic status, and/or access to technology. Techniques such as stakeholder maps may assist in identifying attributes and decomposing stakeholders into more relevant sub-groupings.
- Don’t rely on a single technique – when eliciting information from a large population of stakeholders, consider using multiple techniques across non-mutually exclusive groups. For example, follow-up surveys with interviews, consider confirming results from an observation using a follow-up questionnaire etc.
- Support with hard evidence – where you are soliciting complex or historical information from stakeholders, confirm with hard evidence wherever possible, such as using emails, audit logs, meeting minutes and/or policy documents.
- Census, not survey – where possible and practical, consider making responses to surveys and questionnaires ‘compulsory’, essentially turning them into a census. This approach can also be applied to interviews and observational studies where a stakeholder group is small.
- Use advisory groups – where available, include advisory groups, unions, or other available advocacy group in your elicitation exercise to represent stakeholders. Advisory groups are more likely to have in-depth knowledge about a stakeholder group that may allow them to identify areas analysis may have failed to address. Advisory groups may also be useful when validating the findings of an elicitation exercise and understanding if further elicitation is required.
- Understanding the context – take the time to understand the full context of the situation you are analysing, including the terms, acronyms and phrases that have a specific meaning to stakeholders. Consider keeping a glossary of terms and making sure it is validated by stakeholders to ensure language isn’t taken out-of-context.
- Be self-aware – take the time to reflect on your own personal perspective and how it may influence your analysis. Where you are aware of a personal bias, consider finding and confiding in a colleague who can critique your approach to ensure it remains unbiased. Consider removing yourself from initiatives where you have a clear conflict of interest.
Bias is a risk to any business analysis initiative. However, with a little planning and self-awareness, any risk of bias can quite easily be mitigated or avoided.