post

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

Successful product management in companies where it’s least understood

Measuring critical user journeys – Javier Andrés Bargas-Avila (UX Director, Google)

A new way to look at applied experiments

Product survey findings: Only 15% of users are embracing AI features 

Building high-performing product teams: The PDLC approach

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Guest Post

Overcoming the 20% feature usage challenge

Product Management

Three product leaders to learn from at #mtpcon inside Pendomonium

Product manager, indie publisher, and writer with experience managing software, content, and media products. I also write for CBS Interactive websites, among others, and co-manage SF Product Tank.

Tori Funkhouser

<p>There are some important differences between managing a solutions vs a platform product, in terms of providing product value. One of the key differences is in how you determine and track valuable key performance indicators (KPIs).</p>
<p>This post focuses on the differences for managing KPIs not just for platforms overall, but for content management systems &#8211; products used by web sites to build their content, which include the tools that editors use to publish content to the site.</p>
<p>There are lots and lots of helpful articles on what KPIs are, and how to best determine which KPIs are best for your business (including <a href="https://www.klipfolio.com/blog/best-practices-for-picking-business-kpis#gref">this one</a>, <a href="https://hbr.org/2012/10/the-true-measures-of-success">this one</a>, and <a href="http://www.industryweek.com/lean-six-sigma/five-rules-selecting-best-kpis-drive-operational-improvement">this one</a>). So here is just a quick overview of what KPIs are and how you can go about choosing them.</p>
<h2>How to Choose Your KPIs</h2>
<p>KPIs are a set of measurements meant to indicate whether or not a product is successful. Because so many other articles have written on this subject, this is a quick (oversimplified) plan for one way you could go about choosing KPIs from your product.</p>
<p>This plan borrows ideas from the product innovation model used by Netflix (see its blog post <a href="http://techblog.netflix.com/2011/01/how-we-determine-product-success.html">here</a>) and many other product companies to quickly determine success for product innovations. (Remember: once you choose your KPIs, you should stick with them for a while! This could be a good way to at least get you started.)</p>
<h4>Plan:</h4>
<ol>
<li style="font-weight: 400;">Start with your goals. Don’t try to paste another product’s goals onto your product. Your product and business is unique.</li>
<li style="font-weight: 400;">Come up with a hypothesis for what metrics might allow you to prove that you’re delivering value, according to your goals.</li>
<li style="font-weight: 400;">Brainstorm ways to cheaply implement a test for whether or not the metrics you chose will consistently test for value.</li>
<li style="font-weight: 400;">Repeat the test until you find metrics that work for your product.
<ol>
<li style="font-weight: 400;">Remember: measure metrics that reliably reveal cause and effect. See more on this important point <a href="https://hbr.org/2012/10/the-true-measures-of-success">here</a>.</li>
<li style="font-weight: 400;">Measure only the most important things. Keep the list small (try <a href="http://www.industryweek.com/lean-six-sigma/five-rules-selecting-best-kpis-drive-operational-improvement">three items</a> per category).</li>
</ol>
</li>
<li style="font-weight: 400;">Once you’ve found metrics that reliably test whether your product is providing value, track those metrics over time to see if your product efforts improve (or maintain) success.</li>
<li style="font-weight: 400;">Re-check every so often that your KPIs are still the right indicators of your product’s success.</li>
</ol>
<h2>KPIs for Solutions Products vs Platform Products</h2>
<p>Okay! That’s done. Moving onto the differences between solutions products and platform/CMS KPIs.</p>
<p>Because a [solutions] product is something you sell and a<a href="https://www.quora.com/What-is-the-difference-between-a-product-and-a-platform" target="_blank" rel="noopener noreferrer"> platform is common infrastructure that you build products on</a>, their goals and therefore the ways they define success are, naturally, pretty different.</p>
<p>For instance, solutions products often focus on revenue growth and customer growth as key drivers of product success, so they are measured via KPIs such as increased MRR (monthly recurring revenue added in a given month) or the rate at which prospects are converted to customers. See here for more <a href="http://blog.aha.io/software-product-management-metrics/" target="_blank" rel="noopener noreferrer">metrics ideas and suggestions</a>.</p>
<p>Platform/CMS products are not usually directly involved in generating revenue, so many of the KPIs that solutions product managers might measure are not relevant. That, however, does not mean that platform or CMS products do not have an impact on a business’s bottom line.</p>
<p>So what are some good KPIs for a CMS product manager to think about? For our product, we break our goals into four categories: financial goals, systems goals, user (customer) goals, and innovation goals.</p>
<h3>Financial Goals</h3>
<p>Because I work for a media company with thousands of editors producing content for websites, one of the goals of the CMS is to drive efficient production or publication of the words, images, videos, and other media that make up the site’s content.</p>
<p>Some KPIs for increasing efficiency of content production include:</p>
<ul>
<li style="font-weight: 400;">Percentage decrease in time-to-publish</li>
<li style="font-weight: 400;">Reduction in cost (in employee hours) to program pages</li>
<li style="font-weight: 400;">Reduction in cost (in employee hours) maintaining pages and doors</li>
<li style="font-weight: 400;">Increased number of newly published content items per week/month</li>
</ul>
<p>Also, CMSes (particularly enterprise CMses) are often implemented to save costs &#8211; i.e. in order to not have to pay vendors to handle tasks the company can handle on its own, and to drive efficiency.</p>
<p>A simple way to test whether the CMS is saving costs is to determine the cost of a hosted solution vs the cost of the staff and third-party services. Of course, this cost must also be set against the opportunity cost of allowing for greater custom innovation (see below).</p>
<p>Reporting these KPIs to stakeholders who are funding the CMS, or another enterprise platform, keeps them up-to-date on the value of their financial investment.</p>
<h3>Systems Goals</h3>
<p>Much of what matters to the users of the CMS and to the business depends on the efficiency of the platform.</p>
<p>A CMS product team must work with the engineering management team to quantify data quality, integration efficiency, infrastructure reliability, and other systems-related concerns.</p>
<p>To drive efficiency, a CMS product manager may work with an engineering manager to determine the metrics such as the following for the following categories:</p>
<ul>
<li style="font-weight: 400;">Data quality
<ul>
<li style="font-weight: 400;">Percentage decrease in reported bugs</li>
<li style="font-weight: 400;">Percentage increase in specific types of log errors</li>
</ul>
</li>
<li style="font-weight: 400;">Release efficiency
<ul>
<li style="font-weight: 400;">Decrease in number of bugs created in QA or after release</li>
<li style="font-weight: 400;">Decrease in integration costs (in engineering hours) spent on releases</li>
</ul>
</li>
<li style="font-weight: 400;">Reliability
<ul>
<li style="font-weight: 400;">Percentage decrease in time outages occur</li>
<li style="font-weight: 400;">Percentage decrease in MTBF (mean time between failures)</li>
<li style="font-weight: 400;">Percentage decrease in MTTR (mean time to repair ) &#8211; i.e. consider which failures matter most to users and stakeholders and measure how often they happen, and how long they take to be repaired</li>
</ul>
</li>
</ul>
<p>Working with engineering management to determine and report on KPIs is a vital piece of a CMS or platform product manager’s role.</p>
<h3>Innovation Goals</h3>
<p>Another key reason to use an enterprise CMS is to create custom solutions to support innovative content.</p>
<p>To measure whether or not a CMS is enabling greater innovation, the product management team can measure the percentage increase in the revenues from custom features vs. the cost of producing the feature.</p>
<p>Also, if the goal is that the CMS allows readers on the end site to experience more innovative content, this article: <a href="http://www.strategy2act.com/solutions/content_management_excel.htm" target="_blank" rel="noopener noreferrer">&#8216;Ready-to-use KPIs to Track Content Management Performance&#8217;</a> suggests that product managers can “measure reader&#8217;s attitude to published content” &#8211; i.e. intentionally track reader sentiment or behavior related to custom features enabled by the CMS.</p>
<h3>User (Customer) Goals</h3>
<p>As Jody Roberts says, we “can&#8217;t talk about measuring the success of a CMS, without talking about measuring the success of that CMS&#8217; consumers”. As with all products, the users of that product should be satisfied (or more) with the product. CMS product managers can use simple surveys to measure customer temperature tests over time: are they unhappy, neutral, happy, or very happy with your product?</p>
<p>Then, of course, there’s the handy <a href="https://en.wikipedia.org/wiki/Net_Promoter" target="_blank" rel="noopener noreferrer">Net Promoter Score</a>. Even in the context of an internal CMS, product managers can ask users whether they’d recommend the CMS to new employees (or would encourage the business to continue funding the product).</p>
<h2>Conclusion</h2>
<p>Though the definition of success might be different for solutions vs platform products, a CMS, like any product, needs to provide value for its users (customers) and the business, and be measured against metrics that prove its value.</p>
<p>To do so, a product manager must understand their goals, work with their engineering counterparts and stakeholders to determine the right KPIs to track those goals, and then continually review whether or not the KPIs are really enabling them to report valuable insight on the success of the product.</p>
<p>Whether your platform goals are to increase efficiency, increase data quality, or delight users, it’s key to think of KPIs that will track and show value for <i>your</i> unique product.</p>
<div class="g g-21"><div class="g-single a-170"><a class="gofollow" data-track="MTcwLDIxLDEsNjA=" href="https://www.pendo.io/pendomonium/"><img decoding="async" src="https://www.mindtheproduct.com/wp-content/uploads/2024/06/MTP-Footer_ad.png" /></a></div></div>

core/freeform

Product Management Skills

How to keep your head about generative AI (when everyone is losing theirs) by Claire Woodcock

Measuring KPIs for a Platform vs a Revenue-Generating Product

Tori Funkhouser

Comments

Read next