Democratising Online Controlled Experiments at Booking.com by Lukas Vermeer

BY ARNE KITTLER ON MAY 24, 2018

Booking.com is a European company with a reputation for running experiments in a sophisticated way. Lukas Vermeer is a senior product owner at Booking.com and one of the key players who established not only the tooling, but more importantly the culture behind experimentation at Booking.com. The title of his keynote at MTP Engage refers to an academic paper published by Lukas and his colleagues, but his MTP Engage talk focuses on the cultural aspects of experimentation at Booking.com.

Ideally, tricky product decisions are not just left to product managers or the HiPPOs in the room, but supported by meaningful experimentation.

The problem is that often product managers either don’t feel empowered to run experiments the way the would like to, or they find there’s difficulty in getting the rest of the organisation to understand and believe the results when they run them.

He describes how a centralised and standardised system is a necessary basis for independent, decentralised decision-making.

He emphasises the importance of formulating and testing hypotheses and how a major part of running experiments is challenging assumptions, particularly if your product team is different from the target audience.

Lukas recommends ranking assumptions in order to identify and test the most risky ones as you move forward. He shares a personal story that illustrates how this can lead to completely different solutions from those you in had in mind when setting off to optimise a particular aspect of your product.

About ARNE KITTLER

Director Mobile Product at XING, the leading professional network in the German speaking markets. Co-Organizer of ProductTank Hamburg and Curator of MTP Engage Hamburg.

Get the best product content from across the web

Sign up for our free weekly newsletter with a lovingly curated selection of the best product content from all over the web.

24 Shares
Share
Tweet
Share