In the first of this 2-part series on product research, we look at how to get started with product research and at some golden rules to makes sure it goes well (read Simple steps to effective product research – Part 1: Getting started). I part 2, we look at what commonly goes wrong, how to avoid this, and at best practices for analysing your results.
Where does research typically go wrong/fall short?
Product research is a mix of user research, market research and analytics. It means looking into the problems customers have, understanding them in a market context, and then bringing them with analytics to show you how people are using your product.
You fail to consider all three elements
In practice, though, teams often focus on just two out of these three elements. According to C Todd Lombardo, VP of Product and Experience at Openly: “Product research tries to bring these three streams together, but a lot of the time they’re talked about individually. You might see teams saying, ‘I’ve got my UX research, and I’ve got my analytics’. But they lack the market research and the context that delivers in helping you to describe your product strategy and market.”
The scope isn’t clear
If the precise intent is unclear, then people may get stuck in the weeds, so make sure the scope of the research is crystal clear. Nadia Elinbabi, Senior User Experience Designer at Openly, says she once had a three-month research project turn into a nine-month project because the scope just kept expanding. “If you’re not very clear on the exact answers that you’re looking for, you can be doing it for the rest of your life,” she says.
You conflate research and interviews
People can conflate research questions with interview questions. For example, the research question “what are the best restaurant experiences”, isn’t a question to ask at interviews – whereas “tell me about the last restaurant meal you had” is.
Egos and assumptions get in the way
People’s egos can get in the way of effective research. We might think, thanks to our experience and market knowledge, that we know the right product to build and then fail to conduct proper product research. As C Todd puts it, you need to expect to be wrong, rather than expect to be right. Just because you got the right answer once, doesn’t mean you’ll be right again. This chapter, Prepare to be wrong, from C Todd’s book on product research, looks in more detail at the mindsets that contribute to poor product research
Tips on effectively analysing your results
What do you need to keep in mind when it comes to understanding the results of your product research and getting the most from your analysis? Here are a few pieces of advice from our experts.
Preparation is key
Effective research and analysis is all in the preparation. You need to be completely clear about the purpose of the research and what questions you want to ask before you start. Determine who to ask questions to, and whose data would be valuable.
Keep it moving
You may not always have 100% of an answer. But don’t forget that 80% of an answer may be good enough. As C Todd says: “We want to make sure that we’re moving fast enough, but also in the right direction.”
Make sure the team gets involved
Go through your research with your teammates. The more people involved the better, because the biases you need to be aware of when asking questions also apply to analysing results. Says Nadia: “The more people that you have digging through the research with you and going through the analysis, the more likely you are to get to the real answers.”
What books, talks, and articles do our experts recommend people refer to?
- Just Enough Research – Erika Hall
- Interviewing Users: How to Uncover Compelling Insights – Steve Portigal
- Deploy Empathy: A practical guide to interviewing customers – Michele Hanson
- Talking to Humans: Success starts with understanding your customers – Giff Constable
- The Product Experience – Product Research Rules – C. Todd Lombardo & Aras Bilgen
- Product Research Rules – C Todd Lombardo & Aras Bilgen
- Qualitative Intelligence by Mitchell Gillespie