The median product is dying a slow death. If that seems like an overstatement, then the fact of the matter is that if usage rates are declining, a product is probably – not necessarily, but probably – in trouble. The 2017 Mixpanel Product Benchmarks Report showed that Average Daily Active User (ADAU) growth, in which active users are defined as “active” if they perform any action in a product, was actually negative in two of the four categories we measured: SaaS and media & entertainment.
For better or worse, I’ve had an opportunity or two to learn about what to do when a product is not growing in the way you want it to. There are basically two options: fight or flight. You can either try and make changes to your product, or give up on it.
But first, you need to assess if the usage decline is real, if it is something that is going to adversely affect your business, what is causing it, and what you’re going to do about it.
Is Usage Actually Down? Do you Care?
For some companies, ADAU is simply not a metric that impacts their bottom line. At Mixpanel, we find it useful because it is easy to use and understand and because it correlates clearly to desired business outcomes, but that’s not the case for all companies. Investment or tax tools may care more about monthly or annual usage, and others, such as support products, may even prefer for ADAU to decrease.
But if you do care, there are a lot of legitimate reasons why usage can decline. There are seasonal factors. If you’re a SaaS product and your usage declines over the Thanksgiving break, hopefully you know well enough not to panic. Check in on how seasonal declines compare against the historical data, and if it’s in line with previous seasonal decreases, don’t fret.
If it’s not seasonal factors, though, declining usage is not necessarily the end of the world. Basically: declining usage is fine if you are getting more value from the users who remain. That means if the users who remain are more engaged, or providing more lifetime value. If you’re becoming better at monetizing users, usage declines may be tolerable. If your company is shifting its focus upmarket and making fewer, bigger deals, usage may decline even as revenues go up. It’s a pretty simple multiplication problem: users x user value = $$$$. As long as one part of that equation is increasing proportionally more than the other, there is no problem.
But if that’s not happening, then it’s probably time to take a long hard look at why usage is down and what you can do about it.
Okay, I do Care! What can I do?
If your usage numbers are declining, and it’s not for the reasons discussed above, then you need to figure out why. And to understand what is going on, you should talk to your users, or rather, your ex-users. Understand what their biggest gripes were, what their pain points were, and why they left.
The second question to answer is where the decrease is coming from. ADAU is a metric that is driven by a number of other metrics. The first of these is signups. How many people are coming in the top of the funnel. Then there’s activation. How many users are onboarding to your product? Then there’s both short-term and long-term retention. Of the users that have onboarded, how many are coming back?
And it could be any or all three of those stages that is responsible for the decline. Figure out where the problem is. If you’re seeing signups decline, it’s an entirely different set of actions to try and remedy the problem than if you’re seeing a decline in retention.
If the decline is in signups, here are a few questions to ask:
- Has our marketing changed?
- Does our marketing or messaging need new positioning?
- Are we targeting the right set of users?
There are short-term fixes like email-based marketing, SEO campaigns and demand generation. But in the long run, you need to figure out what message is resonating in the market that will entice customers to even try your product out.
If the issue is in activation, you need to look at your funnels. Ask these questions:
- Is the quality of our signups the same, or is our customer mix of a lower quality than previously?
- What was working well in our onboarding that hasn’t been working as well lately?
Watch your customers go through the onboarding flow and do deep user research.
If the problem is retention, we’re assuming that users have signed up, onboarded, and are just not coming back enough. If that’s the problem, there’s really only one question to ask:
- What’s wrong with my product?
Retention problems are at a product level. If you’re not solving a key pain point for people or providing value, retention will suffer. There’s no quick fix here. If your product is not doing that, or if competitors are doing it better, you’re going to continue to lose users.
With that in mind, let’s look at two times we’ve had this problem at Mixpanel and how we resolved it.
Mixpanel Mobile
Problem: signups, activation, and retention
Mixpanel Mobile usage, July 2016-March 2017
The Mixpanel Mobile app was not growing the way we wanted in early-to-mid 2016. Our Average Daily Active Users (ADAU) number was pretty flat, perhaps tending toward slightly negative. The fact of the matter is, the combination of features available was not enough. As a result, we wanted to keep investing in growing the mobile app and find ways to deliver additional value through it.
So we evaluated a number of different options and ended up launching our Smart Alerts product last November. Smart Alerts sends customers push notifications any time their metrics moved significantly. We gave our customers the option of marking if the alert was useful or not and for the mobile app it was right around 90% useful straight away.
The results from adding Smart Alerts were instantaneous. We saw a huge spike the week we launched, and more importantly, have sustained a double-digit usage boost since then. Our product wasn’t delivering for our customers the way we wanted it to be. So we changed it.
Mixpanel Survey
Problem: activation and retention
Mixpanel Surveys March 2016-April 2017
We used to have a Surveys product where we would allow customers to set up surveys in order to get qualitative feedback from their users on mobile. So for example, let’s say you’re a company using the Mixpanel Mobile app and a customer did a specific action; maybe the customer unsubscribed or cancelled their billing payment. You could then get a survey and ask them if they had any feedback for why they did that action. The goal was to allow our customers to get qualitative information from their customers.
Unfortunately, we never got the kind of traction we wanted from that product. We weren’t seeing the kind of usage growth or retention numbers we needed and it wasn’t driving much business value.
One problem with product development in general is you could build a million different features but then you have to support those million different features. Each of those features will probably be lower quality if you’re invested in too many different things in too many different areas. So with surveys, we weren’t seeing growth and decided the more prudent course of action was to kill it.
We could have invested more engineering effort and product effort and design effort into improving that product, making it more valuable, taking on the various survey tools of the world, but it wasn’t central to our focus and we were just spreading ourselves too thin. It wasn’t worth the investment we would need to put on it. So the decision was made to retire that product.
And that’s always a hard decision. People have worked hard on it. Customers still are using it. It’s a delicate process. It’s imperative to make the withdrawal as seamless for your customers as possible. That means giving them several months’ notice. While we only fully retired the product in March 2017, we notified customers in December, giving them several months to get ready. In our case, it meant providing tools for them to transfer their survey data into other tools. It meant saving historical data so they could still access it. You can’t just turn it off overnight.
Killing a product isn’t fun. But it’s better than letting a zombie product drain resources from your company and stifle potential new and existing products that are actually serving users.
Comments 2