Why you Need a Product Analytics Playbook

BY SPENSER SKATES ON FEBRUARY 16, 2017

When it comes to successfully implementing a data-informed approach to product analytics, it’s easy to find lists of tips and tactics. It’s much harder to put together a strategy you can return to time and again.

Every product is different, of course. There’s not a one-size-fits-all approach. But we do have common methodologies for framing questions and answering them, for figuring out what’s interesting in your data, experimenting, learning, and growing.

Amplitude has taken these best practices from across the industry and collected them in The Product Analytics Playbook. The first volume of the Product Analytics Playbook: Mastering Retention, is released today.

We decided to focus on retention as our first topic because it’s the most critical metric for the success of a business, yet it’s often overlooked in favor of acquisition rate, installs, or daily active users. Here are our big findings:

1. Retention is Different at Every Stage of the User Lifecycle

Too many companies focus only on retaining their new users. But only a percentage of users at any given time can really be said to be “new.” You can’t just think about new users when you have so many who have been using your product for three days, a week, two and a half months, and so on. Each one of those groups has different needs and is in a different context with your product.

There are:

  • New users: Users you’ve just acquired
  • Current users: Users who have been using your product for a defined period of time
  • Resurrected users: Users who became inactive and have returned to your product and become active again

Looking at your user base through the lens of a retention lifecycle means using tactics specific to each group in an attempt to bring all users into an active, current state:

  • You want to activate your new users
  • Reactivate your resurrected users
  • And turn your current users into extra-engaged power users

6 - 3.2 Lifecycle Framework (2 col)

Current user and resurrected user retention are both high-leverage ways to understand your users better and increase your overall retention.

In mobile gaming, there’s a word for power users—whales. Whales, despite being generally very few in number, account for something in the order of 70% of all spending on in-app purchases. Now think about how silly it would be if mobile game developers sought only to “activate” new users and devoted no resources to creating more of these whale gamers. That’s the power of focusing on current user retention.

It’s the same with resurrected user retention. It might seem odd to focus attention towards helping churned users come back and get value out of a product, but do it right and it can be a serious driver of success. Evernote is famous for doing this to particularly brilliant effect:

evernote-smile-graph

Users re-download the app every time they switch devices, producing a “smile” effect as they’re reactivated. With a large number of users, this can be a huge driver of revenue.

Resurrected user retention, current user retention, and new user retention are all important. It’s just that the tactics you use to approach them will be different.

2. You Need a Strategy Organized by Behavioral Persona

Looking at your user base in terms of the entire lifecycle forces you to understand that each segment has different goals and different needs. Each responds to a different kind of motivation and nudging.

When you plan out your product development, you should prioritize features and fixes according to the group of users you’re trying to target.

But current, new, and resurrected aren’t the only ways to segment out your users. You should also segment based on behavior in your product. Behavioral personas organize users into groups based on how they tend to use your product. If we take YouTube as an example, we can identify three distinct groups quite easily:

  • Viewers
  • Creators
  • Viewers + Commenters

Each one of these groups has different needs too, and YouTube has to decide which group is most valuable and important in order properly to prioritize product development.

You probably have different ideas about which behavioral personas are the most important at different stages of a company’s growth. In the early years of Facebook, the most important users may have been those that invited the most of their friends—the product’s greatest vectors for virality. Today, the most important users may be those that share the most content—those who create the engaging Newsfeed experiences that maintain the product’s growth.

Screen Shot 2017-02-10 at 6.30.40 PM

Whoever your most valuable users are, you need to amplify their behavior. It’s about understanding the value that people derive from your product and capitalizing on it.

3. Retention is not What you Think it is

What is an “active user”?

Is it someone who opens your app, or is it someone who opens your app and takes a specific action, or a specific set of actions? Without a meaningful definition, your daily/monthly active user metric is nothing more than a stock vanity metric.

This is why it’s important that you go beyond mere activity and define what we call your app’s critical event. This is the core activity of your app—the tweeting, the posting, the listening, etc.—that marks real usage. Calculating your DAU/MAU through your critical event metric will give you a real-time feedback loop on how well you’re doing at driving engagement with your product.

That’s not the only thing to consider when it comes to tooling. You also need to work out which of the three major methods you should use to calculate your retention in the first place:

  • Unbounded: Shows you what percentage of users come back on a particular day or later
  • N-day: Shows you what percentage of users come back on a particular day
  • Bracket: Shows you what percentage of users come back within certain preset time brackets

4 - 3.1 Bracket Retention

For a grocery delivery app with hefty minimums and big average order sizes, you wouldn’t necessarily expect users to be coming back more than biweekly to place orders. The fact that users open your app once a day is interesting—but likely not really meaningful for your company’s success.

A social app that relies on a constant flow of new content, however, cannot survive if its users are only coming back every two weeks to post updates. It needs to be an integrated, daily part of its users’ lives. Even if they open it a lot, it doesn’t matter unless they’re actually participating and therefore making the experience valuable for others.

The method you choose to measure your retention with needs to align either with your product’s natural usage or with what you hope that usage will become. You need to look at critical event and current usage patterns to find your product usage interval, whether that’s daily, weekly, or monthly—then you can measure retention in a meaningful way to track improvements to your product.

Without retention, your product is a leaky bucket; you can pour in as many dollars as you like into marketing, advertising, and other means of user acquisition, and still wind up with no long-term users—which means no sustainable growth and no means to generate revenue.

There are thousands of articles listing “growth hacks” for increasing retention, ranging from social media tactics, to drip campaigns, targeted ads, and push notifications. The problem with most of these tactics is that they’re just that: short-term, often unspecific, tactics that lead to a transient uptick in active users. To date, we haven’t had a broadly applicable framework for improving user retention at various stages of the user lifecycle.

The Product Analytics Playbook aims to fill this gap. We developed the frameworks in the book through conversations with retention experts and our experience working with some of the leading product and growth teams in the industry — and we’ve validated the methods by applying them to real web and mobile products from different verticals. We’ve also partnered with some of the best companies in the product world to help us present the chapters of the playbook, including:

  • mParticle: customer data control, segmentation, and enrichment
  • ProdPad: product management, roadmapping and backlog software
  • MindTheProduct: 50,000+ member international product community
  • ConversionXL: conversion rate optimization consulting and training
  • Adjust: in-app analytics, advertising attribution and app store statistics
  • Appboy: multichannel app lifecycle marketing, messaging, and metrics

Through eight chapters, this volume of the Product Analytics Playbook will teach you:

  1. The basics of retention, and why it’s so important no matter what kind of business you’re running or customers you’re selling to
  2. How to identify your product’s critical event, or core value proposition
  3. The Retention Lifecycle Framework and how to apply it to your own product
  4. The product analysis toolkit: a collection of methods and analyses for understanding user behavior
  5. Why current user retention matters, and how to learn from your most successful current users to boost overall retention
  6. How to effectively onboard and retain brand new users
  7. How to bring back resurrected users and keep them retained long-term
  8. How to prioritize experiments and apply everything you’ve learned into a cohesive retention strategy

This isn’t about “growth hacking”. It’s about giving you strategies that you can put in place for products in all industries and at all stages of growth.

The Product Analytics Playbook, which we prepared over the course of 2016, clocks in at 155 pages. We’re excited to share it with you, which is why if you’re one of the first 25 to request a download of the PDF, you’ll also receive a free hard copy of the Playbook.

Get your copy of the Playbook here—and happy analysis!

 

Spenser Skates

About SPENSER SKATES

Spenser Skates is the CEO and co-founder of Amplitude. He experienced the need for a better product analytics solution first-hand while developing Sonalight, a text-to-voice app. Out of that need, Spenser created Amplitude so that everyone can learn from user behavior to build better products.

RELATED CONTENT

Read more about Product Management Skills ...

Stepping up Your A/B Tests

161 Shares
Share
Share
+1
Email
Tweet