A/B Testing

As the UK shops itself out of recession, it was more than fitting that February’s ProductTank London was all about product management in commerce: online, offline and mobile.  This month we had three eye-opening talks: CEO at Depop Runar Reistrup (@runarreistrup) talking about mobile design for the real world; “the Malcolm Tucker of testing” Craig Read more »

On a bright, crisp Saturday morning in February, loads of you joined us for our best ProductCamp London yet to share, learn and network with other product people.  We were especially pleased to have product management guru and ProductCamp co-founder Rich Mironov (@RichMironov) joining us from the West Coast. What’s an “Un-conference”, Anyhoo? There were Read more »

Running a multivariate test or even a simple AB test can be daunting. With each variation, there are additional tests to be run and measured, and it can all get overwhelming very quickly.  At ProductTank October, Alex Smith, the Site Development Manager at Charles Tyrwhitt, showed us what he measured and why, and talked about Read more »

Multi-armed bandit testing involves a statistical problem set-up. The most-used example takes a set of slot machines and a gambler who suspects one machine pays out more or more often than the others.  For every token, they need to decide which slot machine to use in order to maximise winnings from their budget. This set-up Read more »

As product managers and entrepreneurs, we know the importance of validating an idea before committing to getting it built. However, validation is easier said than done. In fact, it’s possibly one of the hardest things you will ever do in your product life cycle, otherwise products and startups wouldn’t have the abysmal 90%+ failure rate Read more »

Introduction Many people know they should be doing experiments and testing their product but when starting out it can be daunting working out where to begin. So, below is my handy guide to experimentation built from scars received on the coal face of experimenting. This post will take you through a birds-eye tour of the Read more »

At ProductCamp London in February, Lucy Spence ran a great session on data driven decision making. She shared many of the lessons learned as a UX manager and now Head of Product at LOVEFiLM, all in her inimitable sketchnote style. She covers the gamut from simple A/B testing through Multivariate (MVT) testing and radical A/B Read more »