A/B tests are a common solution to finding more out about new product features and your customers. However, running so many tests a week can lead to a few potential drawbacks. In this weeks’ Sunday Rewind, we look back to when Kevin Shanahan, head of growth at Peak, shared some of the learnings he gained after running 60+ A/B tests at Peak, looking at each step of the A/B testing cycle in turn.
“We went from treating every A/B test as a one-off to being able to run lots of tests in parallel that had a better chance of succeeding. And this process became one of the most important factors in our growth; a repeatable way for us to experiment with different ideas and improve the product.”
In this deep dive into testing, he explained a number of setbacks that could have been prevented if he followed some of his own advice from his future self. This includes brainstorming, prioritising, analysing every data point, and many learnings.