post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

Sunday Rewind: Pivot isn’t a 4-letter word – Sunil Parekh

Artificial intelligence (AI)

Why AI development demands product operating models

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

A guide on implementing effective AI evaluations

The most common skill gap in aspiring product managers — and how to fix it

How to deal with information overload in product management

Meta bets big on AI while Lovable reaches $2 billion valuation: AI news roundup

June’s top product management reads

Alexandre is a Product Manager and Growth Manager, working at Frontier. Frontier designs products and services for distributed and agile organisations: Crystal.work, Straw.work, Yolocracy.org.

Alexandre Gabadou

<p>Facing uncertainty is a company’s biggest challenge. It doesn&#8217;t matter whether you&#8217;re in a startup or an established company, when you start work on a product, you need to validate your problem, your solution, and find your market. When you scale, uncertainty arises in the form of change &#8211; your market changes, your users’ needs change, and your organization changes. With that in mind, the most decisive tool you have when facing uncertainty is knowledge. Some companies rely on instinct and opinions to make decisions, but this article focuses on knowledge as a decision-making model.</p>
<h2>The Data / Information / Knowledge Paradox (Drawn From the <a href="https://en.wikipedia.org/wiki/DIKW_pyramid" target="_blank" rel="noopener noreferrer">DIKW Pyramid</a>)</h2>
<ul>
<li><strong>Data</strong>: data is feedback from your users, your market, or your product. It can be a lengthy spreadsheet (quantitative) or a one-on-one interview with a user (qualitative). Data is an <em>input</em>, and as such has no real value in itself, it tells you what is happening.</li>
<li><strong>Information:</strong> information is the <em>output</em> of data, after analysis. As such, the goal of data analysis is to find out why something is happening. Information is only as good the analyst’s capacity to understand data &#8211; or the UX researcher’s capacity to empathize. In that sense, the gap from data to information is technical. Quantitative data becomes information in the hands of of the data-savvy. Qualitative data requires method and empathy.</li>
<li><strong>Knowledge</strong>: knowledge is the <em>outcome</em> of data. Knowledge only arises when an information is digested and accepted by its recipients. Unfortunately &#8211; though data cannot be refuted &#8211; information is almost always open to doubt. And, if information can be doubted, then it’s up to the subject to accept or not accept the information. Thus, the gap from information to knowledge must be cultural.</li>
</ul>
<div align="center">
<table>
<tbody>
<tr>
<td></td>
<td>Question</td>
<td>People involved</td>
<td>Skills needed</td>
</tr>
<tr>
<td>Data</td>
<td>What is happening?</td>
<td>Individual</td>
<td>Technical</td>
</tr>
<tr>
<td>Information</td>
<td>Why is it happening?</td>
<td>Individual / Team</td>
<td>Analytical</td>
</tr>
<tr>
<td>Knowledge</td>
<td>What have we learned?</td>
<td>Team / company</td>
<td>Organizational</td>
</tr>
</tbody>
</table>
</div>
<p>&nbsp;</p>
<h3>Going From Data to Information: the Learning Curve</h3>
<p>Professionals spend their careers perfecting their expertise on how to extract information from data. Data analysts have software and querying to treat data at scale. UX researchers rely on empathy and method to understand user psychology. Data or user research is often considered a job that requires a high degree of specialization.</p>
<p>However, most people don’t need to achieve perfection to manipulate data. In fact, more and more tools come out every year to bring data in the hands of the data newbies, making it less complicated. User friendly data platforms are making the technical challenge increasingly achievable.</p>
<p>Knowing how to use a tool, however, is only part of the equation, you also need to know what to make of it. The uncertainty in going from data to information can come in several shapes:</p>
<div align="center">
<table>
<tbody>
<tr>
<td></td>
<td>Questions</td>
</tr>
<tr>
<td>Data collection</td>
<td>What data should I collect?<br />
How should I collect it?<br />
Is my data reliable?</td>
</tr>
<tr>
<td>Data analysis</td>
<td>What do I measure?<br />
How many metrics do I monitor?<br />
How can I be sure that this data means that?</td>
</tr>
</tbody>
</table>
</div>
<p>As mentioned before, some people spend their careers learning all of that. Knowing how to apprehend raw data or how to choose the best graph out of an analysis takes time. If you’re the data-savvy one at your company, this is where training, documenting how tools work, demos and so on, come in handy.</p>
<h3>Going From Information to Knowledge</h3>
<p>Extracting information from data often means one person doing the analysis and coming up with results. Going from information to knowledge, however, involves other people &#8211; potentially many. It is exactly because other humans are involved at this stage that we now have to look at company culture:</p>
<ul>
<li>Organizational silos cause information to travel towards the top, and rarely across departments. Not everybody gets the same access to information (unfortunately, as we’ll see, diversity of thought is key).</li>
<li>Managerial layers distort the information as it travels upwards: the decision made at the top can end up being very far from what was intended in the beginning.</li>
<li>Just because you’ve made your analysis available, doesn’t mean that people in your company will hear it. Companies often suffer from <a href="https://en.wikipedia.org/wiki/Confirmation_bias">confirmation bias</a>, especially when the information challenges accepted opinion.</li>
</ul>
<p>The model below offers one approach for success in your endeavor. For now, we can see that if going from data to information requires expertise, going from information to knowledge relies on company culture. Hence the <span style="font-weight: bold;">paradox</span>: a company that leverages data needs to work on its culture.</p>
<h2>The Decentralized Model for Data Governance</h2>
<p>The flow from data to information to knowledge naturally mirrors the flow from the individual to the team to the whole. So, to build your data model, you need to address three levels &#8211; the company, the product and the individual.</p>
<h3>Company Level</h3>
<p>One of the most common mistakes when setting up a data governance model is to over-engineer it and measure everything. Building any kind of governance is deeply iterative. You should start small and work your way up. Not everything needs to be measured, at first. In fact, too much noise can be worse than no data at all. Start by measuring the most important thing: your north star metric (NSM).</p>
<h5>What is a North Star Metric?</h5>
<p>It’s the one metric that measures your company’s impact (Amplitude wrote <a href="https://amplitude.com/blog/2018/03/21/product-north-star-metric">a great article</a> on this). If someone asks you how successful your company is in realizing its mission, what would you think about? For some it can be a revenue metric, others a value metric, but it is NEVER an acquisition metric (ie monthly signups).</p>
<p>Your north star should be</p>
<div align="center">
<table>
<tbody>
<tr>
<td>Automatic</td>
<td>It can be accessed in minutes at any point in time.</td>
</tr>
<tr>
<td>Actionable</td>
<td>Our actions have a measurable impact on it.</td>
</tr>
<tr>
<td>Relative</td>
<td>You can compare it to competitors and through time.</td>
</tr>
<tr>
<td>Fundamental</td>
<td>It is directly linked to your product’s value proposition.</td>
</tr>
</tbody>
</table>
</div>
<p>&nbsp;</p>
<p>Let’s take a for instance, Medium’s NSM: Time spent reading.</p>
<ul>
<li>Automatic: Total time spent reading is something that can be easily displayed, and can be accessed by anyone.</li>
</ul>
<ul>
<li>Actionable: When you see total time spent reading, you very quickly see what to improve. You either have to increase the number of people reading, or the number of people writing. It makes growth very explicit.</li>
<li>Relative: Total time spent reading can be measured each week/month/year and measured in cohorts, to track evolution. It can also be compared to other blogging platforms.</li>
<li>Fundamental: Total time spent reading is baked into Medium’s value proposition.</li>
</ul>
<h5>Why you Need a Macro Metric</h5>
<p>Call it north star metric, call it company goal if you don’t like buzzwords, what matters is what macro metrics bring to the table:</p>
<ul>
<li>Teams often have different objectives. But, when you collectively define a north star metric, you create a common goal that naturally aligns all teams and transcends silos.</li>
<li>When you define a north star metric, teams understand what role they play in the bigger picture and it’s easier to make sense of the overall purpose of the company.</li>
<li>When teams are synchronized and they understand how they can help the company’s mission, there is a lot more trust and increased autonomy.</li>
</ul>
<h3>Product/Team Level: The Funnel Approach</h3>
<p>Once you have set up your macro goal, you are ready to play your part in bringing up that metric. When you look at Medium’s north star metric, you can see how teams play a role in increasing the total time spent reading.</p>
<ul>
<li>You can increase traffic on your platform.</li>
<li>You can work on conversion.</li>
<li>You can get more of the same readers to come back.</li>
<li>You can work on how users share content.</li>
<li>You can work on pricing to get more users engaged.</li>
</ul>
<p>Any of these actions would end up increasing the north star metric : total time spent reading. This is exactly what the AARRR framework is all about.</p>
<ul>
<li><strong>Acquisition</strong>: How do you get more users?</li>
<li><strong>Activation</strong>: How do you get more users engaged?</li>
<li><strong>Retention</strong>: How do you get users to come back?</li>
<li><strong>Referral</strong>: How do you get users to bring more users themselves?</li>
<li><strong>Revenue</strong>: How do you monetize? (if you monetize)</li>
</ul>
<p>Of course, these are macro questions, they’re an abstraction of what happens in the field. However, they are key in changing the way a company approaches growth. In answering the question “how do you get more users engaged?”, you come up with hypotheses. Some hypotheses may have a high degree of uncertainty, some have a high impact on your metric, others little impact.</p>
<p>This is where data, information and knowledge come into play. You start with a set of questions. For each question, you come up with hypotheses. You experiment, gather the data, analyze it, and find an answer to your assumptions. With the new set of information, you come up with new hypotheses, try them out, learn from them, and so on. You go from work based on opinions to work based on hypotheses. And, it is in the process of solving hypotheses that people, teams, and the company learns. Growth becomes highly predicated on knowledge, and faster growth means faster experiments.</p>
<h3>Individual Level</h3>
<p>Of course, coming up with hypotheses isn’t as straightforward as saying that it comes from data analysis. If you put 10 very similar people in front of the same piece of data, it’s likely that you’ll have 10 very similar hypotheses. Coming up with ideas requires diversity of thought, and <a href="https://firstlaw.wordpress.com/2011/10/18/ashbys-law/">Ross Ashby’s law of requisite variety</a> is probably the best example of this need for diversity. Information isn’t one of those things that should be left to the company’s data experts and executives. In fact, the more people participate in the identification and testing of hypotheses, the richer your results will be, and the faster you’ll learn. In plain words, your capacity to learn as a company is determined by the amount and scope of people involved in the experimentation process.</p>
<p>So, what is diversity of thought? Simply put, it’s the extent to which individuals in a company can approach a problem from different perspectives. It can come from the diversity in people &#8211; having people from different genders and backgrounds. It also comes from the diversity in know-how &#8211; working in teams made of different expertise (feature teams, squads etc.). It can also come from involving people at different stages in their careers, for example bringing junior employees together with more senior employees. Place the same set of data in front of a diverse group and you will gather rich and insightful hypotheses on how to achieve your impact.</p>
<h3>How do you Encourage People to get Interested in Data?</h3>
<p>The biggest challenge is technical. The learning curve is pretty flat and can be discouraging. The tools you choose to build your data stack are determinant in the accessibility of data. As such, user-friendly data platforms such as Amplitude, Mixpanel and such can play a big role in making it easier for individuals to jump on board. Such tools can also be very useful playgrounds to get individuals interested in the harder stuff.</p>
<p>Failing the use of simple tools, you can document techniques, tools, or processes, as well as offer to organize group training on different topics. Give open access to the raw data and document its structure. Also, encourage people to ask questions about the data, and improve your documentation accordingly.</p>
<h2>In Conclusion</h2>
<p>Perhaps the hardest part in getting people interested in data, is making them curious to try in the first place. If you have to remember one point from this article is that it isn’t about data, it’s about learning: learning at an individual level, at the team level, and as a company. But, thinking that individuals want to learn in their company implies that they are excited about what they do. Curiosity comes from believing in your company’s purpose and adhering to its culture, and it can’t be forced onto people. If you’ve tried everything above and still, people aren’t interested in data, it isn’t because data’s boring, it’s because your company is.</p>


core/freeform

How Booking.com tackled a data quality crisis: Nimit Bhardwaj (Data Product Manager) – Product Unplugged

How to lead data-centric product teams: Jon Mora (Chief AI Officer, Zefr)

How to be a better data-driven product manager

User insights for product decisions by Cris Valerio

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

Why Data is key in Building a Company That Learns

Alexandre Gabadou

Read more