post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

Career advice from our #mtpcon workshop leaders

Career

Mistakes, learnings, and a product person’s relentless journey into leadership

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

Join us at #mtpcon London in 2025 – mark your calendars

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

Why teams often struggle with OKRs: A conversation with Jeff Gothelf

Advice for internal product managers

Deep dive: Crafting your product vision and mission

<a href="http://www.littledata.io/about-us">Edward</a> has broad experience advising companies on business analytics, and is currently building tools to simplify web analytics at <a href="http://www.littledata.io">LittleData</a>

Edward Upton

<p>Measuring the actions real people take on your digital product should be at the heart of your product development efforts. So are you also investing time in the measurement process itself?</p>
<p>Most of us product managers work with services which are already live, so measuring what has happened already is critical to deciding what to improve next. You need reliable &#8216;eyes&#8217; on your customer&#8217;s behaviour, and to spot if new product deployments or new marketing channels have broken any part of your conversion path.</p>
<p>[Tweet &#8220;Measuring what has happened already is critical to deciding what to improve next&#8221;]</p>
<p>Unfortunately, data collection and reporting does go wrong, and many products I&#8217;ve worked with have been &#8216;blinded&#8217; by interruptions in their analytics. It&#8217;s pretty embarrassing when your boss asks how progress is going on a key goal, and your answer is &#8216;I don&#8217;t know&#8217;.</p>
<p><img loading="lazy" decoding="async" class="alignnone size-full wp-image-5381" src="https://www.mindtheproduct.com/wp-content/uploads/2014/07/Screen-Shot-2014-07-26-at-07.27.42.png" alt="The Dreaded Flat Line" width="776" height="268" srcset="https://www.mindtheproduct.com/wp-content/uploads/2014/07/Screen-Shot-2014-07-26-at-07.27.42.png 776w, https://www.mindtheproduct.com/wp-content/uploads/2014/07/Screen-Shot-2014-07-26-at-07.27.42-300x103.png 300w" sizes="(max-width: 776px) 100vw, 776px" /><strong>The dreaded flat line&#8230;</strong></p>
<p>Here are some tips to make your analytics setup more resilient, and avoid those holes in your data.</p>
<h3>1. Pick less than 6 key metrics</h3>
<p>Honestly, it&#8217;s hard enough to gather good data consistently for one metric. Don&#8217;t complicate this ten-fold by reporting on stuff that no one cares about.</p>
<p>Sit down with your team and agree what metrics you really rely on for your jobs. It&#8217;s tempting to post information on everything if there&#8217;s no cost of collection, and figure out what to pay attention to later. But if your product has been going more than 6 months, it should be possible to pick a small set of useful ones &#8211; and ignore the rest.</p>
<p>Have a look at the <a href="http://www.slideshare.net/Leananalytics/lean-analytics-workshop-from-lean-startup-conf" target="_blank">Lean Analytics</a> approach to picking useful, non-vanity metrics.</p>
<h3>2. Consider what questions you&#8217;d like to ask of your product</h3>
<p>Think about how you are going to use the analytics, and what kind of questions you need answered. Write down a list of the top 10 questions, and make sure you are capturing the right data in the right format to answer your question.</p>
<p>For example, if you want to see whether subscribers under the age of 20 are more active users than those over 20, then you need to post the age for every registered user (this is possible using <a href="http://www.littledata.co.uk/2014/07/what-are-custom-dimensions/" target="_blank">custom dimensions</a> in Google Analytics), and make sure that is a metric accessible in pageview or session reports.</p>
<p>Considering the end reports can also guide which collection and reporting platform to use. Google Analytics may be perfect for that session-by-age-group graphing above, but if you want to look at individual customers you may need <a href="https://www.kissmetrics.com/" target="_blank">KISSMetrics</a> or a <a title="Going beyond pageviews: simple ways to track your customers online" href="https://www.mindtheproduct.com/going-beyond-pageviews-simple-ways-track-customers-online/">custom GA setup</a>.</p>
<h3>3. Ask &#8216;how will we track this&#8217; for every new feature</h3>
<p>Analytics gets out-of-date every time the product changes, so be proactive and consider how a feature will affect your metrics <strong>before</strong> you release it. In true data-driven companies, you wouldn&#8217;t be allowed to release a feature until you could measure its impact.</p>
<p>You could adapt your Agile process so that features are not signed off as complete until a couple of weeks after release, and the analytics impact has been checked. Or just make your <a href="http://www.quora.com/Stripe-company/Does-Stripe-have-product-managers-or-do-engineers-manage-the-products-themselves#" target="_blank">developers responsible for the end-to-end feature testing</a>.</p>
<p>[Tweet &#8220;Ask &#8216;How will we track this?&#8217; for every new feature&#8221;]</p>
<h3>4. Prioritise speed of recovery over speed of setup</h3>
<p>Yes, analytics is boring and it&#8217;s tempting just to get a junior developer or marketing agency to set up your tracking. But when it goes wrong, will you be able to fix it quickly?</p>
<p>Having someone internally who deeply understands how your data collection and reporting works, and ideally can make updates themselves &#8211; rather than add it to the development backlog &#8211; is a big bonus in times of crisis. And that&#8217;s coming from someone who runs an <a href="http://www.littledata.co.uk/consulting/" target="_blank">analytics consultancy</a> helping clients outsource this setup!</p>
<p>I was a skeptic about the value of tag management software for smaller companies, but I now recommend it as standard &#8211; even if the setup is slightly longer. Setting the rules for sending data from your website in a tool like <a href="http://google.com/tagmanager" target="_blank">Google Tag Manager</a> means you can make instant changes to data captured, without the risk of disrupting the actual app performance.</p>
<h3>5. Test, test, test</h3>
<p>The big danger with tracking events on your app is it all happens in the background, and fails silently &#8211; so you won&#8217;t know until you want to get the report.</p>
<p>Have you stepped through each conversion step and checked the appropriate data is being captured?</p>
<p><a href="https://support.google.com/analytics/answer/1638635?hl=en">Real-time reports</a> are essential for testing, but you may need to set up a separate test view or tracker for your test environment if you want to avoid polluting your real customer data.</p>

core/freeform

How to lead data-centric product teams: Jon Mora (Chief AI Officer, Zefr)

How to be a better data-driven product manager

User insights for product decisions by Cris Valerio

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

Analytics Strategy: 5 tips for more reliable reporting

Edward Upton

Read more

Comments