No one is immune to cognitive biases. Cindy Alvarez, Principle Researcher at Microsoft and Author of Lean Customer Development wants us to recognize this fact. Because while we can’t avoid bias altogether, she has some advice for us on how to work around bias and reduce the impact it has in our research.
You are all an incredibly smart audience, and it doesn’t actually matter, because cognitive biases are really going to screw up what you do anyways. Cindy Alvarez
Our brains are not very good at making nuanced decisions. So cognitive bias creeps in and impacts the questions we ask, what people tell us in response, and how we interpret what we have heard. Cindy talks us through several common biases, how they manifest themselves in our work, and what questions we should be asking instead.
Types of Bias
Confirmation Bias
With confirmation bias, we look for evidence that proves us right, and avoid or ignore evidence that contradicts our beliefs. We don’t do this on purpose, but it happens. Instead of asking “How likely would you be to use (a solution)?”, ask “Tell me about what you’re currently doing in (a situation).” Hearing people’s stories is better way to discover what they really need and what they really care about.
Cognitive Dissonance
Cognitive dissonance is the feeling when we’re trying to hold two contradicting views at the same time. When research is not proving what we thought it would, it is tempting to ask, “Are we sure we’re talking to the right customers?” This helps us feel better by saying it’s the users fault, not ours, but it’s not solving the right problem.
Instead, start by saying, “Let’s assume (an undesirable belief) is true. Now what?” Just the semantic break of pretending something is true lets our brains release and helps us get to a different answer.
Survivorship Bias
We want to talk to happy customers because it makes us feel good. This is the heart of survivorship bias, or “Happy Customer Bias.” But your happiest customers are not typically the best representation of what needs to change in your product.
Instead, talk to churned customers, low-usage customers, or non-customers using competitor products. Talk about what drove them away or why they’re not using it. Ask them questions like, “If you HAD to change one thing…”, “Which tasks do you put off doing?”, or “How have you done (a task) differently in the past?”
The Curse of Knowledge
Once we have a fact or skill, we cannot remember what it was like to not have that fact or skill. So the people who have used your product for a long time don’t know what it was like before they started. We’ll often ask “How hard was it to get started with (a solution)?” and you’ll hear people say, “It wasn’t that hard. I figured it out.”
Instead ask, “Suppose you had a new coworker join your team – what advice would you give them to get started?” People will immediately start telling you all of the difficulties and workarounds they have to implement to use your app. When you ask in this way, people don’t feel like they’re criticizing you, they feel like they’re helping a friend.
Social Desirability Bias
When you ask people, “How often do you go to the gym?”, many will say 5 or 7 days a week with a straight face even if they only wish they exercised that often. This is social desirability bias – we (unconsciously) edit what we say to make ourselves look good. So think about if your questions have a socially-preferable answer that might bias your results. For example, “Do you have a hard time meeting deadlines?” Who wants to say yes to that?
You would get more honest information by asking, “In the past month, tell me about a time when you got something done later than you originally promised.” We give people permission to admit to faults, which helps us see problems to solve.
Backfire Effect
The backfire effect – presenting rational evidence against our beliefs can make us reject it, and believe in our original thought even more strongly. Or put simply: you can’t fight feelings with facts. If a customer thinks they need a feature, you can’t convince them otherwise.
To manage the backfire effect, don’t tell someone “You’re wrong.” Say, “You’re right – and…” Nothing deflates a fight better than “you’re right” or “I’m sorry.”
Get to the Real Answers
Cindy wraps up by talking us through an example of asking questions in the right way to avoid common biases. She gave us some key questions that will help you understand if your customers truly need what they are asking for:
- “Just to be sure I’m clear – if you had that already, how would it make your job/life easier?”
- “Since you don’t have it today, what is your current workaround?”
- “In the last 6 months, how often has that happened?”
- “You’re right, (our competitor) does offer this feature. I’d love to hear your impression of the solution.”
- “You mentioned it might come in handy in the future. How are you anticipating your needs changing?”
- “If you could wave a magic wand and change anything about (a situation), it doesn’t have to be possible, what would it be?”
These types of questions help you understand if there are actually problems to be solved and if the features your customers are asking for are really needed, or if they are just things that are nice-to-haves that can be removed from your roadmap. Asking questions in the right way will help you avoid biases, understand your customers better, and build the things that are really needed to solve problems.
