Is your Product Underperforming? "Product people - Product managers, product designers, UX designers, UX researchers, Business analysts, developers, makers & entrepreneurs October 10 2020 True Data-Driven, Data-Driven Product Management, Experiment, Experimentation, Kpis, Performance, product hypotheses, Mind the Product Mind the Product Ltd 1115 Photo by Kolleen Gladden on Unsplash Product Management 4.46

Is your Product Underperforming?

BY ON

As a long time Product Manager, it was with a certain trepidation that I accepted a promotion to become Head of Data at tutti.ch. tutti.ch is a free-to-sell-on classifieds platform based in Zürich, Switzerland. The role came with the hopes and expectations to unlock value from the data we collect from the millions of customers a month that come to buy and sell on our platform.

One thing brought up by many before I started was to sort out our Key Performance Indicators (KPIs). We had changed these at least once a year and couldn’t settle on a real structure. There was a strong feeling that other companies were much better in this aspect. After much research, I pieced together bits of advice alongside my own thoughts and I hope this can be a help to you.

Performance vs Technique

Consider Usain Bolt. He runs fast and is fuelled by chicken nuggets.

Usain Bolt – 2009 IAAF World Athletics Finals. Photo by Ververidis Vasilis

He participated in many athletics events but excelled at 100m, 200m and 4x100m relay. This required a huge amount of training and focus but how could he have made sure what he was doing was correct?

Performance KPIs

Well, he had his performances, which are the outcomes from all of his training. But even these had a whole bunch of things he could have looked at:

  • Personal Best (PB) time
  • Season Best (SB) time
  • Average time from past 3 races
  • Top 3 finishes
  • Gold Medals
  • World Ranking

None of these is perfect on their own. PB time only changes if he beats it, it doesn’t show when he is in a slump. Top 3 finishes isn’t so great when you get so used to gold medals that silver seems like a loss. World Ranking is good but what if you want to be a legend of the sport with records that will stand for decades?

Picking the Performance KPIs that are right for you depend on what your goals are and you still most likely need a set of them to get around each other’s weaknesses. Even so, the smaller the better, 3-5 performance metrics is best for companies to get a clear enough picture in an easy to understand package.

OK, so Usain (and hopefully you) are now clear on what success is; improve those Performance KPIs. But how? What should Usain focus his training on to get better? He and his trainers get together and discuss. Usain thinks he needs more chicken nuggets to give him more fuel for his runs. His trainers aren’t so sure.

One of the trainers suggests they look to understand the biomechanics of running and proposes that the speed you run is governed by a simple formula:

Step Length x Step Rate

Everyone agrees, but another trainer interrupts “How can we tell if we have hit the best balance between those?” GOOD QUESTION! They could look at videos from races to get those metrics for each of the other sprinters and see where they lay (aka benchmarking). They can run their own experiments to push Usain further than he would naturally go.

Technique KPIs

But these metrics, while informative, still aren’t to the correct level where they are actionable. You can try running with a longer step length than feels natural but chances are that your step rate goes down accordingly. So the team comes up with a plan, they believe that Usain could improve his step length by improving his technique with great hip flexibility and calf strength and his step rate through better arm swinging and longer quality sleep for recovery. It must be said, they are not positive that these are the cause but they are using their experience to guide them as best as possible and as they increasingly learn they will adapt.

What about product companies?

Performance KPIs

Do you really think you are all that different from Usain Bolt and his trainers? Ok maybe a bit, but the same principles apply. First, find out what your Performance KPIs are. You need to look for where the key moments of value are for your customers and your business. This should come quite easily as it will be the main actions for your product, here are some ideas:

Customer Performance Metrics

  • Items sold
  • Rides taken
  • Lessons completed
  • Emails sent
  • % of ratings given 5 stars
  • Gross Merchandising Value

Business Performance Metrics

  • Subscriptions started
  • Subscription churn rate
  • Advertising revenue

For tutti.ch, I did this by mapping out our main customer journeys first and just picking the moments of most value as you can see below (value points in blue).

Map of a customer journey
Map of a customer journey

Technique KPIs

Once you have these, you need to work out your Technique KPIs for improving them and this is where you need to bring your knowledge and expertise. Discuss with stakeholders of the performance metrics and identify what you believe is needed for success.

For instance, for our buyers customers segment, we believed that we needed:

  1. A steady stream of users coming that don’t bounce
  2. Good quality search results
  3. The impulse to contact the seller about the item
  4. Them to think of tutti.ch again soon for another purchase

We then paired each of these beliefs of what is important to corresponding Technique KPIs to get:

  1. Number of active buyers
  2. % of searches that a top 3 result is clicked on
  3. Contact rate
  4. Buyer churn rate

A word of warning: if you have mostly metrics that you will see in a standard web analytics tool (users, sessions, page views etc) then you should really consider if changing them will bring the impact that you desire. These are often called vanity metrics and rightly so as they make no direct difference to the value that you are creating. It’s ok to have one or two to contextualise other numbers but keep them few and far between. Focus on behavioural metrics that are core to your product and how it creates value.

Recap

Remember, first establish your moments of value and measure how often they occur or how much value they generate to get your Performance KPIs. Then use the expertise within your company to plan out the techniques that go into improving each of the Performance KPIs as it’s those you take action upon. Next, you need to think about how to present and report your KPIs and use them to generate validated learnings, but you’ll have to wait for the next post for advice on that!

Best of luck and let me know how you get on!