Hopefully, we all understand how important it is to talk to customers. As Marty Cagan observed in an interview last year, they’re one of the main sources of insights about a product, helping us to understand the issues with our products and why users do or don’t use them.
But what if you’ve never conducted a user interview, or you don’t feel confident doing so, or you never seem to get useful results from your user interviews? What do you need to know to help you navigate a way through them and ensure success?
This guide is intended to complement Mind the Product’s training module on user interviews, and which has lots of detail and practical exercises on types of interview and how to use interviews as decision-making tools. This article should give you some insights and tips on how to run the interview process effectively. It looks at:
- How you might choose who to interview
- The problem of bias and how to manage it
- How to shape effective questions
- Becoming more confident as an interviewer
- How to process your interview data
Choosing who to Interview
There are two groups of people you can interview – one is those who use your product, the other is those who don’t. Whether you interview the first or the second, or a combination of the two naturally depends on what you’re trying to understand. Says Mind the Product’s managing director Emily Tate: “If you’re doing a bigger discovery effort then it’s helpful to have a bit of both. People who use your products have incorporated them into their workflow, and into their lives. People who are not, either because they use a different product or have no product knowledge will give you a different understanding of the pain points when people look at a problem.” If you’re interviewing non-users you should try to be specific about the persona you’re trying to build a product for, she says, because “if you’re trying to build a product that’s focused on working moms, it doesn’t really help to get a college student to do your user interview”. For more on user personas, visit the training module or see these posts from Mind the Product trainers Stephanie Musat and Roisi Proven, How to Write Effective User Personas for a Diverse Audience and Why you Should Create a Person, not Just a Persona.
There are plenty of companies who will, relatively cheaply, do some of the heavy lifting for you and source and schedule interviews, although you’ll have to devise some screening questions to narrow down the types of people you speak to. However, as Emily points out, you need to be on the lookout for people who are effectively professional testers or interviewees, or who are only there for a promised incentive. “You have to expect that you’ll have at least one low quality interview, regardless of where you get people from,” she says.
And how many people should you interview? Dilip Chetan, a leader in UX research who has built and led research teams in companies ranging from large enterprises to startups, says that five interviews should be the minimum. “It’s five per segment,” he says, “that’s the lowest I can recommend.” Emily expands: “The rule of five is true-ish. I tell people that when you stop learning new information, you’re probably done. For example I was once building a product for buyers of hospital supplies, a very specific market with a very specific workflow, so we quickly arrived at themes. But interviews for say, Spotify, would be very different, because the scope is so much wider. If you’re not getting consistent insights out of the first five people, then you need to interview more.”
If and when it’s manageable, says Dilip, then conducting interviews face-to-face in the customer’s location can be very valuable. “Let’s say you’re a small business owner, and I come into your premises. I can see the pace with which you handle your business as you talk to me. It tells me so much about the unspoken environment and the kind of challenges you face.” Emily adds that you shouldn’t expect to conduct every interview in-person, and points out that “if you do then you’re limiting your pool of people to those who are physically close to you, and therefore introducing bias to your sample”.
Video is preferable to an audio-only interview. Says Emily: “Often you’re interviewing strangers, so video helps you to build rapport. Your interviewees can see your face and when you’re writing notes. If you’re just on the phone, that can be an anxiety producing moment, because they don’t know why you’re being silent.” But remember that interviewees should feel relaxed and well-intentioned towards you. Emily’s hospital supplies purchasers, for example, tended to be a little older and weren’t always tech savvy. At the time she felt that video interviews would have been counterproductive, though now, through Covid, we’ve all become much more used to video calls so this might no longer be the case.
Both Emily and Dilip recommend always having a note-taker with you, or failing that always to record your interviews (with permission from your interviewee). This allows you to more easily engage with the interviewee and build a rapport. “You need to be actively listening and figuring out how to ask the next question, and you can’t do that if you’re taking notes,” says Emily.
Managing the Interview
The User Interviews training module contains detailed advice and practical exercises on the components of an interview, the different types of interview, and building interview questions and how to structure an interview.
This ProductTank talk from Rosemary King, Getting the Most from Your User Research, contains some tips on the art of not inserting yourself into the conversation between the customer and the product, and looks at how to overcome some common mistakes when interviewing.
If you’re new to interviewing or have other people sitting in on the interview there can be a temptation to turn the interview into a sales call. This should be resisted, it’s not the time to sell anything, nor is it customer service conversation. You may well find that customers come in with issues that they want someone from the company to hear. In such cases Emily recommends that you let the customer have their say, and let them know that they’ve been heard, but that you don’t try to solve their problem. “I’ve recommended many times that, regardless of your role within the product, you claim you’re an outside person doing the interview.”
This can be a really helpful tactic because, as Emily says, “people have a hard time telling you your baby’s ugly”. Positioning yourself as someone who is there only to collect honest feedback gives the interviewee permission to say what they really think.
Similarly, if you’re new to interviewing, make sure you get feedback on your technique from someone with experience. An experienced interviewer will easily recognise the things that you do that shift the conversation or fail to set the right tone. Says Emily: “I learned how to interview well by watching people who were good at it. I started to imitate them and to work through what made them so good.” Both Dilip and Emily say that the most important thing you can do is make your interviewee feel calm, so that they want to talk to you. “The best interviewers leave a ton of space,” says Emily, “they’re really adept at letting the silences happen, and letting the interviewee fill the silence.” Dilip adds: “The Holy Grail of an interview is to go beyond just vanilla answers to a question.”
Remember that the purpose is to listen and understand rather than provide any insight, so it’s important you avoid giving the interviewee any sense that there might be right or wrong answers to your questions. People unconsciously look for validation that they’re doing or saying the right thing, so you may need to rein in your natural impulses and temper your reactions to what they say.
A skilled interviewer can ask a question that moves towards the answer they’re looking for without asking the question directly, says Emily. “You wouldn’t ask ‘do you want a pony?’,” she jokes, “you would start with a question like ‘what do you think about animals?’ even if it means you talk about bunnies for 15 minutes.” Crafting interview scripts and questions is a lengthy process that requires careful thought and planning. “Take the time to play with different versions of an interview script, get a second opinion on it. Find people in the product community who can review the script with you and see if they have any other thoughts,” says Emily. It takes work, but the results will be their own reward.
Be Aware of Your Bias
It’s essential to constantly push back against bias when you interview users. This great talk by Cindy Alvarez from #mtpcon San Francisco, Cognitive Biases & The Questions you Shouldn’t be Asking, is a useful run-through of types of bias and how you can reduce your bias by asking the right sort of questions.
As Dilip points out, the first step towards managing your biases is being aware of them. He frames it as soul searching: “Look at the data and ask, ‘why did I speak more with this person? Why did this person turn me off? Or what about this person is not working out for me? And is that fair or unfair’?” Emily adds that when you look back at who you’ve interviewed, you should also ask who you haven’t interviewed. “You need to recruit a diverse set of participants so that you don’t allow bias to creep in. Otherwise you might quickly converge around a theme, and later find that you’ve only picked it because it confirms your biases.” From seatbelts to facial recognition software, we can all think of products where bias during development has had an adverse impact.
Becoming More Confident
They say practice makes perfect, and while we’re not all immediately adept at putting people and their ease and getting them to talk, we can all learn how to do it well. There are plenty of training courses available to guide you through this, and this post, Oh the Drama! What Product Managers can Learn From Actors, from actor Alison Kemp provides some helpful insights on how to become more confident.
Dilip advises the people he trains to spend 10 minutes in meditation before they start interviewing. “It calms your mind before you start an interview,” he says, “and I do it myself. It clears unwanted thoughts and helps me focus on what I want to achieve. Most importantly, it makes things quiet for me so that I’m more receptive to what the person in front of me tells me.” If you’d like to give this a try but are new to meditation check out this quick guide from Headspace.
What then for him makes a successful interviewer? “They’re goal driven, and they know exactly what they want to get out of the interview. They’re very people focused, they’re able to read the situation and tailor their questions or are able to take control without seeming to take control. They’re able to guide the conversation in a pleasant manner that is smooth, and doesn’t jar.”
Processing Interview Data
Raw interview data may take a few forms – detailed notes, transcriptions, video recordings and so on – but whatever form it takes it must be turned into insights. Dilip suggests a debriefing with team members after each interview, because “you have to treat interviews not as an aggregate, but as individual cases”, and sometimes you will find you have to pivot.
“Regardless of the form of your raw data, you want to pull out the insights and valuable pieces of information – the things people said, the observations you made, whatever it is. I like to put them up on a wall and then start grouping them,” says Emily. “The way you group them depends on what you’re trying to learn.” Ideally you should do this exercise with other people, she adds, because we all bring our own biases and experiences to our interpretation of that we hear: “I’ve had situations where different people have interpreted an interviewee’s comments in different ways – one may see a statement as positive, someone else may see it as negative. We listen for the nuggets of information that validate what we already want to do, and we over-index on the things that fit our current views.”
Tips for Success
Interviewing customers is a hard but essential part of a product manager’s role. You have to have a clear goal in mind, project professionalism and empathy, control of the interview while not appearing to do so, make the interviewee feel comfortable and valued even if it turns out they have little that is useful to contribute. Here are a few tips for success.
- Sit in on interviews conducted by more experienced people. They will help you learn.
- Try out your questions beforehand. Refine them till you’re happy that they’re a good fit with your goals.
- Take time to focus before you start an interview. Try Dilip’s suggestion of 10 minutes of meditation and see if it helps you.
- Present yourself as an outsider. If you present yourself as someone who is there just to get feedback then your interviewee will feel they have permission to be honest.
- Don’t try to take detailed notes if you’re asking questions. You need to be actively listening and you can’t do that if you’re taking notes. Have a note-taker with you, or – if you can’t avoid being on your own – make sure you record the interview.
- It’s okay to go off-script. Don’t panic if your interviewee takes you away from your script. There will be opportunities to get back on track. Allow your interviewee to vent if it looks like they need to.
- It takes practice to interview well, so practise and get feedback from your peers. Practise asking the right sort of questions. Sometimes you won’t need feedback, you’ll see immediately that you need to do things differently next time.
Further Resources
- Just Enough Research, Erika Hall
- Interviewing Users, Steve Portigal
- Product Research Rules, C. Todd Lombardo and Aras Bilgen
- >User Research Interviews – Robert Chokr on The Product Experience
- Quantifying Qualitative Research by Leisa Reichelt
Comments 0