post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Podcast

Why should we care about building accessible products? – Dee Miller (Director, Product Strategy & Insights, Adobe)

Product Management

Get better at making better products: Vidya Dinamani, Founder and Partner at Product Rebels

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

A guide on implementing effective AI evaluations

The most common skill gap in aspiring product managers — and how to fix it

How to deal with information overload in product management

Meta bets big on AI while Lovable reaches $2 billion valuation: AI news roundup

June’s top product management reads

Connecting families through food at 🍅Pona, after 10 years in edtech at Edulift.

Co-Founder & CEO

Philip Seifi

While matchmaking is a core lever of any marketplace business, data crunched by Uber, Airbnb, and other marketplaces shows that engineering an optimal marketplace sometimes defies common sense.
For example. You finish dinner and are ready to head home. You see plenty of Uber drivers, but you&#8217;re matched with a car 10 minutes away. It makes no sense!
<h2>Greedy and Network-Optimized Matching</h2>
In our Uber example above, the intuitive approach is to match each rider with the closest car, on a first-come, first-served basis. However, while such transaction-level optimization leads to the shortest possible time for customers in high-trafficked areas, it can also lead to a terrible experience for other customers and suboptimal aggregate wait times.
A better long-term solution, that takes into account the needs of the entire user base, is to predict imminent demand, and optimize matchmaking for a network-level variable such as aggregate wait time. This is exactly what Uber does, in a feature called <a href="https://marketplace.uber.com/matching" target="_blank" rel="noopener” noopener noreferrer">batched matching</a>.
Let’s consider the example below:
<img decoding="async" src="https://www.mindtheproduct.com/wp-content/uploads/2019/10/cd55b4ab-ea9b-405d-ae47-5d2caa10d219_s_52F5273ED0FADBDF795087A2682527BC4DFD91C536A95569D8499970904BB815_1568008355371_image.png" alt="" />
The figure on the left illustrates greedy matching, the intuitive expectation of most riders that find themselves in our scenario. This approach leads to the minimal wait time of just one minute for rider A, but makes rider B wait six minutes, for an aggregate wait time of seven minutes.
The figure on the right illustrates network-optimized matching, which is closer to how most ride-hailing apps assign riders to their pool of drivers. This approach leads to a longer, but still reasonable wait time of three minutes for rider A, and a short wait time of just two minutes for rider B, for an aggregate wait time of five minutes, a 33% improvement over greedy matching.
And since 2017, if a better match becomes available after you book a ride, Uber will automatically swap your trip — a feature internally branded as <a href="https://www.youtube.com/watch?v=a4NuzcYA3pc" target="”_blank”" rel="”noopener” noopener noreferrer">Trip Swap</a>. This may be a nuisance to drivers when it first happens to them, but over the lifetime of their driving will earn them more money.
<h2>Fungibility and its Impact on Matchmaking</h2>
In a fungible marketplace, supply is interchangeable. Ignoring premium options, one Uber driver is broadly equivalent to another. Similarly, most customers will care little whether their Instacart carrot delivery comes from Aldi or Costco.
In a non-fungible marketplace, each supplier provides unique value. Airbnb hosts offer a very different experience from one another. Similarly, on the <a href="https://www.pona.app/" target="”_blank”" rel="”noopener” noopener noreferrer">home-cooked food marketplace</a> I’m currently launching, Pona, where we have home chefs who provide specialist dishes that are not available from other sellers on the platform. In reality, fungibility is not black and white, and whether a supplier is interchangeable with another can depend hugely on market expectations. Many businesses operate between these two extremes (think tradespeople on TaskRabbit, or business formation lawyers on LegalZoom).
Recognizing where your marketplace startup falls on the fungibility spectrum is crucial. It has major implications on how you approach matching, pricing, and other core aspects of your marketplace business.
The Airbnb recommendation engine is a good illustration of how optimal matching changes with fungible supply.
Most &#8211; especially high-value &#8211; customers won’t browse through pages of listings if the initial results fail to inspire confidence in the platform. It is, therefore, a priority for Airbnb to optimize the first five to 10 listings in your search results.
The greedy approach to ordering Airbnb listings would be to show properties that will result in an immediate booking by the user. Under this scenario, all customers would see broadly similar results with an exceptional cost-benefit ratio, and book them on a first-come, first-served basis.
Of course, there’s a better solution. Just as with our Uber example, Airbnb should, and most probably does, optimize search results for network-level variables (Airbnb’s website only says that there are &#8220;more than 50 factors that contribute to each set of search results”).
<img decoding="async" src="https://www.mindtheproduct.com/wp-content/uploads/2019/10/60b85e7b-f166-45d0-9ccb-daf0ef9f67da_marketplace-matching-02.png" alt="" />
In the example above, if a property rental website optimizes at a transaction level, the first customer to make a booking can end up with a better property than they expected. That’s all good and well, but it may mean that a more picky customer making a booking just minutes later is left with no acceptable options.
If the website optimizes at a network level, on the other hand, the search algorithm could nudge the first user to book a different property that&#8217;s equally satisfactory for them and keep the exceptional option available for the picky customer who makes a booking about the same time.
Rather than “will the user make a booking?”, the Airbnb product team might ask something closer to “will showing these five properties lead to the highest aggregate number of bookings in this city?”. They might even optimize for aggregate Lifetime Value (LTV), taking into account that some customers are less price-sensitive than others.
<h2>The Impact of Bias</h2>
Why don’t companies such as Uber and Airbnb prioritize best-behaved, repeat customers in their matching? Wouldn’t it be a good idea for Uber to reward its best customers with faster service? Or for Airbnb to highlight Superhosts above all other results? Just as with greedy matching, bias in matchmaking can lead to suboptimal outcomes at a network level.
This is what we found out at Pona. Most of our cooks specialize in regional, international, and experimental cuisine that is not sold in restaurants, which makes our supply less fungible than for other delivery apps. For example, we only have one Ethiopian home cook in our service area, and she is also one of our top-rated and best-selling chefs. She can only cook a limited number of dishes per day, and always sells out.
On the demand side, the greedy, unbiased approach would be to display her profile to all customers, ordering search results either randomly, or based on distance. A better, network-optimized algorithm might nudge buyers to try food from a different chef each day, or take into account how picky each customer is about their food choices.
In either case, there is a strong temptation to introduce bias into the search results. For example, we could show our top chefs to high-value customers first, as a reward for their loyalty.
Bias can also creep in as a side-effect of a seemingly unrelated feature. For example, some of our customers have asked for a way to follow their favorite chefs, so they can see them at the top of their search results. But what seems like an obvious, and rather benign little feature has the potential to wreak havoc at the network level.
<img decoding="async" src="https://www.mindtheproduct.com/wp-content/uploads/2019/10/3de65181-d950-4bc9-8145-cdcb3134fa6a_s_52F5273ED0FADBDF795087A2682527BC4DFD91C536A95569D8499970904BB815_1568021606776_image.png" alt="" />
Just imagine if a dozen customers &#8220;favorite&#8221; our Ethiopian chef. If we had surplus demand, as in the illustration above, our Ethiopian chef and other top sellers would make all the sales. This would prompt new entrants to leave and in turn, reduce the variety of food available on the platform. If we had surplus demand, new Pona users would never get to see our Ethiopian chef, or taste her cuisine, and would be banished to new and less exceptional sellers.
An unbiased approach allows us to ensure sales are more evenly distributed across all our chefs. If you’ve ever wished that Airbnb allowed filtering based on user reviews, now you have a better idea as to why the feature is unlikely to be introduced any time soon.
<img decoding="async" src="https://www.mindtheproduct.com/wp-content/uploads/2019/10/70d9c771-5f58-46da-b208-a622ad0987ad_s_52F5273ED0FADBDF795087A2682527BC4DFD91C536A95569D8499970904BB815_1568023766804_image.png" alt="" />
To further complicate matters, Pona orders are made at least 24 hours in advance, and our sellers have no overhead costs such as rent or staff. Our chefs say they would prefer to cook one large batch of soup, sell it all on a single day, and enjoy their free time for the rest of the week, rather than sell one portion every day of the week (even if it meant fewer sales overall).
In a surplus supply scenario illustrated above, the ideal matching algorithm would cluster sales on a different day for each cook, which would be impossible if search results were biased by favorites, or a ratings-based ordering.
I know all this can be overwhelming, and as a startup with limited resources you can’t have a team of data scientists optimizing your every match. Fortunately, top academics and product teams at Uber, Airbnb and the like have done a lot of the work for you.
Although each marketplace is different, and you will want to run your own simulations and A/B tests down the line, it’s worth keeping the considerations above in mind as you build your product. Developing a matching algorithm? Think about whether distance is the best variable to base it on. Launching a new feature? Consider how it might inadvertently bias your search results. And as you grow, make sure to dedicate ample resources to these questions.
This article was in part inspired by a talk by Uber product manager<a href="https://medium.com/@tanvisurti"> </a><a href="https://medium.com/@tanvisurti" target="”_blank”" rel="”noopener” noopener noreferrer">Tanvi Surti</a>

core/freeform

How Booking.com tackled a data quality crisis: Nimit Bhardwaj (Data Product Manager) – Product Unplugged

How to lead data-centric product teams: Jon Mora (Chief AI Officer, Zefr)

How to be a better data-driven product manager

User insights for product decisions by Cris Valerio

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

Optimal Matching for Marketplace Startups and the Role of Bias

Philip Seifi

Read more