Evidence-Based Product Backlogs, by John Pagonis "Product people - Product managers, product designers, UX designers, UX researchers, Business analysts, developers, makers & entrepreneurs December 12 2020 True Derisking, evidence, Lean, Lean Product Development, Lean Product Management, Prioritisation, Product Backlog, Product Roadmapping, Validated Learning, Mind the Product Mind the Product Ltd 437 Evidence Based Product Backlogs by John Pagonis Product Management 1.748

Evidence-Based Product Backlogs, by John Pagonis


In this ProductTank London talk, John Pagonis –  then UX Lead at ‘The Mortgage Works’/Founder of Zanshin labs, shares his experience of creating an evidence-based backlog.

Watch the video to see the talk in full, or read on for an overview of his key points:

  • Product Waste – Do you really have time and money to waste building what is not needed?
  • Product requirements are all assumptions – transform your product backlog into a list of assumptions and test your riskiest one first.
  • If your access to users is limited, start with internal user-facing staff such as the support or sales team.
  • Create outcome-based KPIs for each item on your backlog and measure the usefulness.

Why You Need an Evidence-Based Backlog

Since the 70’s statistics show that most software is never or rarely used. Review the usage of your current product and identify any parts which are rarely used. Without knowing which features will be used you are essentially creating waste.

Most backlog items are created using requirements given by internal stakeholders based on opinion, whether that is the opinion of the highest paid person (HiPPOs), the product manager or other colleagues. An opinion is similar to a bet, it is a gamble and you should ask yourself if you can really afford to take that risk with your product.

Backlog items should be meeting the needs of the user as well as the business, so which one should you prioritise? “Start with the customer and work backwards” – Jeff Bezos – if it’s good for the customer it’s likely good for business.

Instead of crafting a backlog based on opinions, you should seek evidence from the one stakeholder who is rarely in the room, the user.

How To Get the Evidence You Need

First of all, you should turn your backlog items into assumptions, ask yourself if they need to be built rather than how or by when.

Assign a value and a risk level to each of your backlog items and test your riskiest assumption first.

For some products, getting access to users in a timely and affordable way can be tricky. Start with internal customer-facing staff such as your support team, sales team or analytics tools and synthesise the data you have first.

Measuring Your Success

Once you have your evidenced-based backlog, attribute each item to an outcome-based KPI. This way you are not measuring the volume of what your team is building but how useful it is. Usefulness can be measured by the usability (how easy or hard it is to use) and the utility (whether it is required or not).