Mention the phrase product management, and your first thought might be of project plans, feature roadmaps, business data, and all sorts of tools and techniques that revolve around a product already in market. In reality, the best and most challenging work for any product team lies in the months before market, during product discovery. And at Mind the Product 2012, London’s inaugural conference for product teams, Marty Cagan, founder of the Silicon Valley Product Group, cut to the heart of successful product discovery.
Start with a vision, continue with passion, be driven by knowledge
As with everything in life, you get out what you put in. So it’s no surprise that Cagan’s top rule is to begin with vision and passion. We’re talking the sort of vision that extends beyond short term results — the kind that takes at least two or three years to get going. But passion and vision without applied wisdom will get you nowhere fast. Make a conscious effort to nail this and you’ll avoid what Cagan says is probably the single biggest reason for failed product.
“People think they can learn what to build from our customers, but of course, you can’t, in technology especially. Number one, because customers don’t know what’s possible because technology moves so fast. Our job is to know what’s possible. Customers don’t — they think they do but they don’t. They define the world based on what they already understand we can do. Number two, and this is more profound, because it applies to all of us: none of us know what we want until after we see it.
“The whole idea of defining a big thing that we’re going to build and launch is a problem. That’s not to say we don’t talk to our customers; just the opposite. The way we overcome this is by talking to a lot of customers. Most teams I work with are talking to more customers in a week than some do in a year. But unlike the marketing mindset where you ask the customers questions, we’re testing on them; we’re seeing if our ideas actually work.”
Testing ideas is important. Ideas, after all, are ten a penny. But in product discovery, as with any creative endeavour, run with the first ideas that strike you as a solution to a problem and you’ll run into problems of your own before too long.
The ‘build and launch’ approach to product discovery costs too much in time, money and lost opportunity. But too often product teams find themselves entrenched in this approach, purely because they can’t test their ideas quickly and cheaply.
Most of your ideas suck.
Deal with it.
Cagan’s approach to product discovery rests on two rules of thumb:
1. At least two-thirds of ideas are never going to work
2. It will take at least three or four iterations to get the remaining ideas to work.
Ideas don’t work, he says, most often because the customer doesn’t care. “We thought they’d love it but they don’t care. Sometimes its because it’s so complicated, it’s just not worth the trouble. That happens. Sometimes it’s just too expensive in terms of time and money to build – so even though we could do it, it’s just not worth it to us. I know that if [the teams I work with] are not killing at least half their ideas, then what they’re really doing is building and launching only to find out that it doesn’t work. Our job is to get through that cycle and separate the good from the bad as quickly as possible.”
It can be very focusing for teams to think of their goal as ‘time to money’ rather than ‘time to market’. For behemoths like Microsoft, that runway could be anything up to three years. But most of us have a runway of about 3 months in which to achieve something significant.
“In a startup it’s seed funding; in a big company, it’s management patience. As soon as they run out of patience, we’re done. So the clock is ticking to come up with a solution fast.”
Iterate and validate, at least once a day.
“There’s a video I like to show people about a week in the lives of a Nordstrom product discovery team, that gives a great example of the rhythm and pace of product discovery. And for most of us, that means at least an iteration a day, so the notion of doing a sprint, a 2-week sprint, even if it’s just one sprint to test out an idea, that’s an order of magnitude too slow. When we’re talking about product discovery we’re talking about the fastest, cheapest ways to validate our ideas.”
Validation can start anywhere from lo-fi prototypes mocked up in Balsamiq to high fidelity prototypes to live data prototypes. Cagan took pains to emphasise the roughness of a live data prototype in comparison to production code.
“A live data prototype is just enough code to send some traffic there, to get some data. Because live data prototyping is awesome for proving that an idea works, empirically.”
In Cagan’s experience, a live data prototype represents about 20 −30% of the cost of real code and really is the bare bones of the idea without the rigour of test automation, comprehensive use cases, or any of the performance, scalability, SEO or internationalisation work that it takes to build a product.
“We just do enough to see how people really behave. When it doesn’t behave well, which happens a lot, when it doesn’t behave the way we hoped it would, then we do user tests.”
“When you A/B test, the data won’t show us why people don’t like things, so a critical technique in our bag of techniques is the user test, where we’re really trying to answer two fundamental questions: Could they use it? And the real question: would they use it? And if they won’t, why not?”
Great products come from great product teams.
Too many product organisations wrong-foot themselves right out the gate, because they make the mistake of thinking of products as projects.
“This is not how you innovate and not how you get to a great product. You have to move away from the project thinking and instead take the approach of having dedicated product teams. First of all, this means cross-functional — product, design, engineering. Second, if at all humanly possible, co-located teams that are sitting right next to each other. This is one of the lowest tech but highest value things you can do, is put them right next to each other.”
“The other side of it is this; you can’t create a dedicated team and then give them a top-down feature-driven roadmap of development. This yields much worse results than the alternative, which is to give that team a set of outcomes that you’re asking them to figure out. Instead of telling them the features in a roadmap, tell them the KPIs that they’re being measured against. What is the result you’re hoping to see? if you give that team these objectives, there’s no guarantee that they will rise to the occasion but it is much more likely that they will because it’s now their product. Their job is to figure out the best way to do it.”
In a good product team, everyone contributes equally to product discovery.
“Products today – it’s not some product manager that has an epiphany. That’s not how it works. Product, design, engineering, [work] side by side. In fact, I’m not afraid to say the little-known secret in product is that the single biggest source of innovation, consistently, is not the product manager, is not even the designer. The biggest source of innovation is typically the engineers, especially when the lead engineer is participating in product discovery. And the reason for that is because they best understand the technology and what’s possible. I argue that if you’re just using your developers to code, you’re only getting half their value.”
Product discovery is about culture, not about process.
“Teams use all kinds of processes – I don’t care about it. There are certain principles that are important; we need rapid iteration, we need frequent small releases, we need things like that. But as far as process goes, honestly, it’s mostly religion. What matters is the culture that you’re putting into place. Let’s say the product manager and the designer disagree, which if they’re good people and they’re doing their jobs well, they will often disagree. Probably the most common disagreement I see is between engineering and design, because there’s often very much a trade off between the optimal experience and how expensive that’s going to be to build out. So what do you do? In a good product culture, you don’t have endless meetings, you don’t have a bunch of escalations; you test.”
“Or maybe the product team says it’s great, but the CEO says no, I don’t like it. This is very common. Bad teams might just roll over — good teams test. We need to be able to run these tests very quickly in product discovery. There’s another common principle behind good product culture, which is that data beats opinions. It’s one of the mantras of Google and I love that.”
“So product culture is a collection of things, but it just means you’re going to move fast, you’re going to constantly learn, you know it’s all about validated learning, it’s not just somebody’s opinion, we’re actually collecting [something] – especially this idea of speed. I’m not saying that you need to work more hours. Most teams are just not working smart.”
Know when to pivot and when to give up.
“I used to talk about pivots all the time, but then we really started to see a problem we bumped into before with teams who tend to give up so quickly. They’re giving up too early. Our job is to be stubborn on vision, but flexible on details. But sometimes, we have to distinguish vision from illusion; sometimes, it’s just never gonna happen.
“If you do the product discovery, first of all, the most common thing you’ll find is that, okay, [this product is] promising, but we need to iterate and get it better so that it actually does what we need. So the most common situation is that you iterate; that’s why I said you want to do those at least every day.
“The second common situation is you decide it’s just not gonna happen. So you discard that idea and you go to the next and the next, you’ve got lots of them. But sometimes when you go face to face with your users, you really understand, you just get very lucky, and you realise, you know, if we relax, maybe loosen up our original vision a bit, maybe we change the target customer, maybe we change the problem we’re trying to solve, maybe we change the business model, you realise… we could have something here. That’s a true pivot. We love when that happens, it doesn’t happen as much as we wish, but when it happens, it’s awesome.”
“The sad thing is so many companies, and honestly so many product people, they have blinders on. It’s very intentional; they are not interested in a pivot opportunity, they are just interested in running their idea into the ground.”
Watch the video of Marty Cagan’s talk.