When I first started hiring business analysts, there was a while there where I was great at hiring BAs that didn’t quite fit what I was hiring for. As a BA myself, I recognized the need to pivot beyond a verbal interview to get a real-time look at a candidate’s skillset. I needed to see what a candidate could do and evaluate that against what was needed for the role I was hiring for.
Enter the Case Study.
Designing the perfect case study is a lesson in trial and error, with each candidate revealing ways to make it better. More than ten years later, I’m still running case study interviews (and have hired some of the absolute best BAs I’ve ever worked with) and have a few tips for hiring managers that want to design their own.
- Keep it simple. It’s a high-pressure situation, do not give your candidate a page of text. You will miss your mark due to misinterpretations, assumptions, over-analysis, time-crunch. A simple picture, a simple task. It’s enough.
- Pick a couple key skills to evaluate. You’re only going to get so much out of the case study without making it overly complex. What are the two or three key things you’re looking for and design to see those skills in action.
- Do not send the case study out ahead of time. You’re just asking for someone to show up with a ten foot long, professionally printed flow chart. Yes. This happened.
- Be VERY VERY clear. Be specific about what you’re asking the candidate to do or produce. Are you expecting them to interview you, draw a diagram, write a requirement? No matter how clear you think it is, someone will misinterpret it. Repeat your ask multiple times.
- Make it relevant to you. Use simple scenarios from real life; industry or domain relevant, something your candidate may face on the job. This will give make it real for both you and the candidate.
- Be flexible. Everyone will interpret your case study differently based on past experiences. Don’t look for your perfect answer, and don’t design with right or wrong in mind. Instead look for skills being used, the questions being asked, the way ideas are communicated; capabilities are the key.
- Make it fun. If you’re stressed, they’re stressed. Keep it casual, make a joke, lighten up. Turn down the pressure to allow your candidates to battle their nerves and show their skills off better.
There are innumerable ways to go about evaluating for a particular skill through a case study. Here are a few examples:
- Facilitation and/or Elicitation Skills. Give the candidate a basic scenario – even something as familiar as ordering a pizza online. Ask them to gather requirements from their client (you) for a new feature. You should predefine your feature with a couple alternate scenarios or edge cases. See how well your candidate elicits your requirements. Did they catch those alternate scenarios?
- Modeling. After gathering the requirements during the facilitation portion of the case study, ask your candidate to create a simple flow chart, use case diagram, context diagram – whatever you’re looking for. Did they accurately model the requirements? Do they know how to communicate using models?
- Working with ambiguity. Include very little detail in your scenario. Throw them into the project with a paragraph and a pat on the back (totally unrealistic, right?). Ask – how would you start? What would you do? Look for how the candidate thinks and how they plan to work knowing very little. Did they talk about identifying stakeholders? Understanding the objectives, benefits or KPIs? Making a requirements management plan? Did they approach the unknown in a way that made you confident they could work through it?
- Strategic thinking, risk analysis, problem solving. Throw a couple blockers into the scenario. The clients are swamped and rarely answer emails. Technical stakeholders are distributed across multiple departments and have competing priorities. The vendor hasn’t provided a data spec for the extract you need to document. The timeline for the project has been moved up and you’re being asked to compress requirements timelines. Don’t be afraid to throw in real scenarios you deal with regularly. I’ve gotten good ideas on problems I was dealing with myself!
In all the above case study scenarios, you can design the challenge and evaluate the output differently depending on the skill level of the role being hired for. For a junior practitioner, I would evaluate a flow chart with a different lens than someone with 10 years of experience. I would expect a senior BA to identify more subtle risks and have tighter mitigation strategies than someone two years on the job.
The case study can be adapted for anything!
As you continue your case study journey, you will continuously tweak, adapt, and perfect. I’ve gotten it so close to where I want it that I enjoy seeing how else a candidate can trip me up and force me to adapt yet again.
My latest tweak: virtual case studies during pandemic times means you must send your candidates a copy of the case study. A digital copy can be forward to recruiters who then give it to other candidates. Candidates then magically produce a ten-page requirements document in half an hour. It took me a couple interviews to figure out that magic. Tweak time! Now there are variations of the case study used at random.
However, you integrate case studies into your process, above everything else, keep it simple and don’t take yourself too seriously. You’d probably find a case study interview stressful too! For those of you on the receiving end of a case study interview – don’t panic. Have some fun with it and show what you can do knowing that the hiring manager’s expectations will be more closely aligned to your skill set upon hiring and you’ve set yourself up to be more successful in the role in the long run.