Soufi Esmaeilzadeh is product director for AI experiences in Google Search, where she leads AI Overviews and AI Mode. In 15 years at Google, she has watched search move from ten blue links to multi-turn conversation.
She joins Randy Silver to explain how her team prototypes and ships at Google’s scale, why she runs lean “SWAT team” pods, and how she balances speed against the quality bar users expect from search.
Key takeaways
- Users changed their behaviour first. Once natural language models let people drop keywords, queries grew longer and more specific, and follow-up questions became an expectation rather than a feature request.
- Soufi frames AI in search as an evolution. Positioning it as a better way to serve the user’s existing need, rather than a threat to the core product, made it easier to win internal support.
- Her team goes from hypothesis to scrappy prototype to real-data demo to a small beta as fast as possible. Debating in decks is the thing she most wants to avoid, because these are not one-way doors.
- Fast prototyping makes the hard question “is it worth it?” Ideas are cheap, so product managers must weigh people, attention and tokens against other opportunities, and build well, not just quickly.
- Quality bars can make or break a launch. Gen UI simulations and audio-over-image explanations landed because the team defined “good” early. A subpar first version would have fallen flat.
- Soufi runs lean pods: one senior strategic lead and a small bench of ICs, with flat lines of communication. Extra layers strip out context, and without context it is hard to move fast and ship well.
- Product basics matter more at speed. Clear hypotheses, shared goals, agreed metrics, and a joint definition of what good and not-good look like still come first.
- A leader’s job in a fast-moving team is to clear the noise. Urgency energises people. Chaos, spinning and cross-team debate drain them.
- For hiring, Soufi looks for three things: consumer product instinct, close attention to where the underlying technology is heading, and a zero-to-one builder mentality.
- AI tools accelerate the work, but the team keeps challenging and cross-checking their output. Sharing where each person has hit the limits is a standing conversation, because those limits keep shifting.
- Org design needs stable problem areas even when the technology shifts. Accountability over an area for two to three years lets people build deep opinions about what success looks like, while their methods change every few months.
Chapters
- 00:00 Introduction
- 01:28 Soufi’s role and background
- 03:09 How AI has changed search
- 05:30 How AI experiences in search evolved
- 07:08 Delivering AI answers at Google’s speed and scale
- 09:04 Evolution, not disruption
- 12:16 What AI means for publishers
- 14:25 Overpersonalisation and quality feedback
- 16:40 Testing and betas at Google’s scale
- 20:40 Gen UI, simulations and visualisations
- 23:39 Balancing cost and ambition
- 25:30 Structuring AI product teams
- 28:21 Keeping quality high as pace increases
- 31:07 Changing processes and measuring speed
- 33:47 Avoiding burnout and context switching
- 36:10 Hiring and promoting
- 37:53 Working with AI tools and where they fall short
- 40:27 Designing teams for the long term
- 42:52 Wrap-up
Episode transcript
Soufi Esmaeilzadeh 0:00
People who have this builder startup mentality, this excitement of getting things done really quickly, of making it happen, of pushing the boundaries and moving at speed is something that I really seek in the DNA of the of the PMs that come and join the team.
Randy Silver 0:21
Hey, welcome back to the product experience. I'm Randy Silver. And you know, there's one of these things that you do a dozen times a day. It's usually the first thing I do when I log onto my computer in the morning is I go to the search bar, uh the address bar on my browser, I type something in, and I'm too lazy to actually do the dot-com. So I get this search experience back. And yeah, I use Google. Uh, a lot of us do. It's, you know, one of these things that has become a copyright defensible word uh because it's not just search, it is Google things. But as the world is changing, as this experience is changing so much, we thought we'd get someone who has a really deep experience in this uh on today. So we've got Sufi Asmalzide, she's the product director for AI experience of Google Search. And welcome, Sufi. Thanks for joining us today.
Soufi Esmaeilzadeh 1:12
Thank you for having me.
Randy Silver 1:14
So I I said what your title is. Do you mind giving everyone a little bit of an introduction? What does that actually mean? Product director uh for AI experiences on Google Search. And how did you get into this world in the first place?
Soufi Esmaeilzadeh 1:28
Uh yeah, of course. It is a uh uh math load. So let me describe it. I drive our AI experiences in search. What that really means is what people know as that box on the top of the search page called AI overviews and the AI mode experience uh you may have heard of or you may have uh actually gone into when you ask a follow-up question, really focused on how do we make those world-class consumer experiences for people whenever they use search. And uh I would argue I have one of the best jobs of the company because I have the privilege of thinking about how our billions of users who use search can really, with so much less effort, get to the heart of what they're looking for, ask what's on their mind without using keywords. Um, and so every morning, I think top of mind for me is how do we continue to make and evolve search to really meet the moment and help people in their day-to-day. And how I got here, by background, I studied engineering. I always loved to build. Um, and then I discovered this thing called product management, which was the perfect combination of gosh, my engineering side that wants to make things, combined with like really pushing that creative strategic side that I love, which is really thinking about what is the right thing to build and what really will resonate with consumers. And so I really fell in love with product management and have um have stuck to it since.
Randy Silver 2:58
And how long have you been at Google?
Soufi Esmaeilzadeh 3:00
Oh my God, I'm gonna age myself. I am coming on to no, I actually uh fully did now 15 years at Google.
Randy Silver 3:09
Fantastic. Okay, so you've seen this change dramatically, and you know, for the longest time, search was one of those things that was it was a solved problem. It was a mature product, at least from the customer experience out of it. I know engineering-wise, there was always room for improvement, but we knew what search looked like uh as a mature thing. What's changed? You know, did AI has come in the fur that it's not 10 blue links at the top of the page. It's not, we're not arguing about advertising links anymore. Now we're talking about the AI, the first return is the AI experience. How did this come about? What do you think about it? How did it how did it start?
Soufi Esmaeilzadeh 3:49
Yeah, so listen, one of the reasons, first of all, that it's kept me at Google for 15 years is because the job is constantly interesting, and it's constantly interesting because we're constantly evolving. So, in one way, I would say, listen, even search has been evolving over the trajectory of history. Like you said, it started as 10 blue links, and then the page did continue to change and change and change. However, I think, Randy, you're pointing out that with this AI paradigm shift, it is opening up an entirely new class of opportunities that we didn't have before. Like things that you would dream up as a product manager are now actually possible to do. Things you wouldn't even thought of your creativity is now getting and going. And so I think in this moment, it is one of the biggest evolutionary moments of Google search where our mission is sort of timeless. This idea of organizing information, of satisfying human curiosity, we're never going to be done. But now we really have the opportunity to push the boundaries on helping people just speak to us in a really natural way, helping people actually engage search, not as one-off keywords, which has traditionally been how people have been using search, but as a multi-turn conversation and something you can come back to and pick up where you've left off. And so it's just creating a whole new class of opportunities on how we can help people.
Randy Silver 5:12
How did this come to you as you know, you this position you're in, it didn't start, you know, one day they someone just decided it had to be there. How did it come to you? How did you end up in this position? And how did the the AI experience of Google search evolve over the last couple of years?
Soufi Esmaeilzadeh 5:30
Yeah. So, you know, it's a combination of, and I think this is how we show up, but product managers every day now, like seeing what the underlying technology can do. We really, I remember in the early days, went to like first principles, where you start really thinking about your hypotheses of how can you actually change the way you help people? Why would that resonate? What do you think are the indicators that you're really helping them out? And I'm going back to first principles because that was the time when we thought, gosh, you know, with the fact that you have these natural language models, can't you let people stop using keywords and just formulate in sentences? And let's try to get that product out there. And what would that product look like? And we built that product and, you know, very quickly, and that's what's amazing about all this. You start seeing consumer behavior shifting. They're starting to ask longer questions. They're starting to say things like, not hotel in uh Los Angeles, but I'm going uh on a trip with my kid to celebrate his birthday, and I'm looking for a hotel that is uh kid friendly to him and has amenities, but is also relaxing for me and my partner. Nobody used to talk to us that way. So you saw that shift starting to happen. And I think building on top of that, when we then start people using that product, we start seeing people's expectations evolve even more. Well, wait a minute, why can't I then ask a follow-up? Why can't I predictably in search get this AI experience when we want it? And so that's when you're continuously building off the cue of also your customers to keep evolving and evolving where this is all going.
Randy Silver 7:08
So when I do this in say Claude or ChatGPT or even in Gemini, there's a lag between the time I ask the question and the time it's thinking to generate the answer. But search, we're expecting the answer instantaneously. And one of the things I've always been really impressed with when I when I use Google search is how quickly that answer comes back. And it's it's not something you can pre-cache in the same way that you would with the most common search results, because I'm guessing the queries are very specific, they're very different, they have to be personalized. How are you addressing something, uh challenge at that scale?
Soufi Esmaeilzadeh 7:45
Well, and I and I'd love if you give me a moment to answer that in a couple of angles. Um because, first of all, the uh first thing that I think is magical about search in this AI era is we recognize that people have all sorts of questions, simple questions, hard questions, and they shouldn't have to think. They should go into the search box, they should ask their question. So, first of all, I think what makes search so magical in this is the capability to ask, answer the simple questions like what's the weather today, to the much more complicated that you raise. Um the second thing that I think is amazing in the search AI era is how do you do that effectively combining the best of the web and AI, because that's the other expectation people have. And the third is, you know, Google does really bring AI at scale to billions of users. It is one of our core differentiators. And I think what you're hitting on is we have a phenomenal uh team of engineers with whom we work to ensure we are building products that can have the reach that search has. We are building products that can answer at the speed at which our users expected. It's just a foundational expectation of search. And so that is some of the technical challenges we are constantly grappling with and optimizing for as we're building our products.
Randy Silver 9:04
Okay, so let's let's stay in the search domain for a moment, but this is an interesting one because it's you are the dominant player by far for search on the web. And we've seen people with the dominant market position any number of times, whether it's Kodak, whether it's Blockbuster, whoever it might be, uh have the problem of not disrupting themselves, not not updating. And even in search, I mean, I'm old enough to remember Yahoo as a ranked directory and not actually be uh migrating to technology that was doing the what the page ranks and things like Google and other things like Alta Vista and Ask Jeeves and everything else falling by the wayside over the years. So, how do you approach this? How do you avoid the problem? How does Google give you the power and what are you doing there and what kind of challenges do you have to ensure that you've got the most valuable real estate in the company? And how are you using that to disrupt and but not lose the the dominance that you have and not lose the financial impact that search has?
Soufi Esmaeilzadeh 10:11
Yeah, I think um the heart of what you're asking me is uh to me, listen, search, uh, how do you evolve it? It is uh you're you're saying it's a very ingrained product. How do you evolve this uh this product?
Randy Silver 10:23
Both technically and and then politically within the and strategically within the company.
Soufi Esmaeilzadeh 10:27
Yeah. You know, I think that um I was saying to you earlier, like part of the reason I've stayed at Google for so long, you asked me the duration, is because I fundably believe this company puts the user first. Uh, we constantly think about the consumer and what they want, and uh that is what we honor. And the reason I'm saying that is then in that is the effect of I don't view this as a disruption, but I view this as an evolutionary moment. And it's a great evolutionary moment for search. It is a moment at which we can really ask how we can do better for our consumers. And I think when positioned that way, when um these discussions were happening and we're saying, hey, listen, this AI paradigm shift is actually going to help us answer questions we couldn't before, get to the heart of what people are looking at we couldn't before. Um, it really paved the way to then demonstrate that. And very quickly, we saw the fact that, yeah, it consumers, it resonated with them. They are using the product more. They are shifting the way they are engaging with the product. And so for me, that's just an indication of gosh, we're on the right track. Um, and hence from there, we then went to building AI mode and uh have this relentless focus around like next, we really want to build your personal information agent. Like that is the future that we're going to. And it's still search at the at the heart of it. What makes search search every day to me is the fact that we are here to organize the world's information. That mission is timeless. So as long as we're honoring really helping people get to the heart of what they're looking for and doing that better, that's the right product. And so I really don't see it as a disruption, but much more as a, gosh, this is the next stage of a product that has been evolving, but for sure we should acknowledge this phase, the degree of and speed of innovation is much more rapid than what we've seen before.
Randy Silver 12:16
I'm I'm gonna challenge you on the disruption and evolution words specifically. Because I can totally understand for where you sit, it's absolutely an evolution. It's an evolution of the experience. You want to delight your customers, uh, you want to give them the answer in as short a time as possible and satisfy them as much as possible. And that is makes all the sense in the world to me. But that information ultimately is coming from other places. There are content providers, and as someone who was a journalist in the past life who is making content now, the difference between SEO and GEO is massive. And the experience of you used to uh send people to me to get the information, you would give them a tease of it and they would come to me to get the final answer. But now you're quoting me and providing the answer, and not necessarily I'm I'm not necessarily seeing directly the benefit of that, or I'm not getting the click through from it. So it's a two-sided marketplace. You're absolutely evolving for the consumer side, but for the provider side, it's got to feel like a disruption. So how do you deal with that?
Soufi Esmaeilzadeh 13:24
Or how do you uh in turn evolve the provider side? No, it's a great question. Listen, I think uh the heart of why people come to search, and we still see this today, is they want to connect to the best of the web. They want to connect to unique voices, they want to connect to authoritative sources, and that's critical in our product development. When we talk about AI experiences in search, we are constantly talking about how to weave in the web ecosystem more effectively into the response and how to enhance better connections for people. So, for example, Randy, we just uh recently launched news and I in our AI overviews and AI mode products. We are constantly investing. I think you saw us talk about this, really bringing in web content via links in the response. And then on top of that, these models, because they're issuing more fan-outs and asking more questions to organize information better, the result is that we can connect people to the actual publisher or unique voice that they want more effectively.
Randy Silver 14:25
One of the other interesting things is, and I recorded another episode last night with someone, and he was talking about the fact that paradoxically, sometimes a lot of the LLMs, the more that you interact with them, the worse the experience gets. The more they know about you, the worse it feels sometimes. How do you keep the experience improving? As you, as you said, with personal intelligence. You know more, it should get better, it should feel more like a trusted companion or a friend, but most of the time it doesn't. What how do you how do you deal with that?
Soufi Esmaeilzadeh 14:56
So I I assume what he was referencing, though I can't put myself in that person's shoe is perhaps overpersonalization. Um, is that what he was getting at, Randy? You think?
Randy Silver 15:06
I think we we went by it fairly quickly, and it was something I didn't really push him on because it was tangential to the rest of the conversation, but I've definitely seen it where there's so much information and there's too much context, or it's it ends up going down a very specific rabbit hole and losing the broader perspective.
Soufi Esmaeilzadeh 15:25
Yeah.
Randy Silver 15:25
Um, so and things like that.
Soufi Esmaeilzadeh 15:27
Yeah. Okay. And I'll probably qualify that under the umbrella of overpersonalization. That can happen, right? Like making associations that are not relevant. I'm assuming that's what they're referencing. Listen, I think that this is where we really get into a combination of like traditional quality hill climbing combined with actually having trusted testers use the product and tell us where the failures are. Like we're constantly actually receiving feedback, whether it's from uh public users giving us feedback through thumbs up, thumbs down, whether it's from actual like beta testers that are using the product and giving us feedback. And so uh that's how we continuously collect signals, especially something like personalization, where you need real users to use the product and tell you yes, no, and they'll do that. They'll do that through thumbs, they'll do that with a follow-up statement, like you got this completely wrong. Uh, you'll do that with actual user studies. So we use all of these various methodologies to constantly quality hill climb the product and ensure that that doesn't happen. Or let me put it this way, ensure we're continuously progressing because it's it is hard to say perhaps it never happens, but that we are iteratively getting better and better at it.
Randy Silver 16:40
Let's talk a little bit more about the the approach that you have to beta testing. Because most of my career I've worked with people who are either in B2B or in more niche B2C. So what we're doing is a very specific solving a very specific problem. And we can test very specific slices or narrow goals. But this is the most wide open thing in the world. This is tell me anything, and I need want to organize the world's information. How do you approach testing and ensuring you have a good experience with something that that wide, that general general I'm not, you know what? Let's just say that wide, because I'm not gonna get the word out.
Soufi Esmaeilzadeh 17:20
Um, yeah, I I love this question because I think when I think about even my organization, Randy, there's uh two tracks to talk about. There's I I'll talk about more of the traditional, and then there's more of the very iterative frontier. Uh, and both of those require different things. I think what I mean in the more traditional is you have a product out there. There are billions of people using it. You continuously need to make that product better. And that's where there is an obsession over identifying places where the quality is not right, um, watching people use the product and every friction point that they're hitting, understanding that, and just maintaining a constant bar of excellence and craft that we have to hit. And then there's the second bucket, which is you have an idea and you're like, I don't know if this is a good idea, but I think there's a there there and it'll resonate. Um, and I think in the world we are today, what I tell my team is you should go from idea to define it to prototype to demo with real data to if there's a there there path to beta really quickly. Do not spend weeks and weeks debating and putting decks together. And we're in a world where you can, you should always anchor on hypotheses, like why do you think this matters? Why do you think this has real opportunity? That early thinking is worthwhile. But now you can spin up a prototype really quickly to even say, hey, this is what my idea is, this is how I think you could connect it to real data and scrappy demo, just the bare minimum of what you need, very quickly gives you the sense of what's easier hard about this? Does this feel good? Is the data completely wrong? Um, and that either kills it and there's no there there, or starts getting the momentum and the organization of wow, I think there's something, and then all the way very quickly to like a beta hand full of testers that can that can feel it out. And so I really think about making sure that that process is happening quickly. I think one of the worst things to do is to spin and lose cycles in debate. Uh, these are not one-way doors. So that's really something on that second track I like to see happen really quickly. Um, of course, uh, like I said, early thinking is important because you should really ask yourself, do I think this is a big opportunity? Do I think this matters? Will it really make a difference in people's lives?
Randy Silver 19:40
Quick one for the product folks listening. AI can draft the PRD, prototype the idea, and help you build almost anything. But knowing what to build, that's the hard part. JIRA Product Discovery helps product teams bring customer insights and ideas together, prioritize what's worth building as a team, and create living roadmaps that connect directly to delivery in Jira. So when more ideas are possible than ever, your team can focus on the ones that matter most. Try Jira Product Discovery for free today at atlassian.com slash MTP. I want to dig deeper into your team and your process, but let's use a concrete example, if you don't mind. Is there something that's happened, say, in the last few months, that was uh a decision that you and the team made that based on user feedback, based on that experimentation, based on the beta, that was really surprising to you that went against uh what your hypothesis was or what you expected?
Soufi Esmaeilzadeh 20:40
Yeah, so I think um if I think about some of the experiences we've launched recently, you know, we talked about uh Gen UI coming into the product and the uh advent of simulators and visualizations. That is a uh skunk group that first had this there, there of we should invest in here and we think there's an opportunity. Um, but very quickly it went that from a talented, small group of folks saying, hey, these are the kinds of use cases that we think if we could create a simulation, it would really change the way that people would think about a product. And uh from there, you start seeing a little momentum and in a clear definition of what good is. And since getting that product out, it has truly resonated with users. But the time from ideation to actual launch was pretty quick. And you saw that it was it was just a clearer way for people to understand uh a concept because they can interact with it, because they could visualize it. That's something we never did in search before. Uh and so um that's an example.
Randy Silver 21:43
What kind of things were you simulating?
Soufi Esmaeilzadeh 21:44
What kind of experiences were I mean, everything from like at what angle should I do a basketball shot to, you know, uh explain to me the rate at which if I switch sort of my financing investment, it could affect my long term uh Savings to sort of examples there of like how you can toggle and play with elements. Um, similarly, there's another one that is near and dear to my heart, but like this idea of can we overlay audio on top of images? And we've now launched that and it's like really resonating and explaining complex uh concepts to people. What is photosynthesis? It's a whole new way. It's as if you have a teacher in the classroom. And these are the kinds of ideas that I think are pushing the boundaries. Um, that as we got out there, we're getting really quick early feedback from consumers that it's responding to them.
Randy Silver 22:32
Okay, so those the way you explain them, they sound like no-brainers, but were they when they first came up, or were there any that came back that were really surprising and challenged the team's expectations?
Soufi Esmaeilzadeh 22:43
I think that all of these things in Notion, you look at it now, you're like, slam dunk, we should do it. But what's really the challenge of these things is how difficult is it to do and can you scale it out is often the question, right? Like it sounds really interesting because you do it at a high quality bar. What does even a high quality bar look like that resonates? That's where the early thinking really matters. Like simulations, I totally get you. You look at that and you're like, yeah, who doesn't want to play with a simulator to a basketball chart? Well, first of all, it's not obvious in search if they would want to do that. But second of all, I think the reason that these products landed is because the quality of the simulation, the definition of good was really high, the craft that we put out there. If we put out a subpar version of it, uh, which is the early days as you're developing in order to get there, the product would have fallen flat. And so really understanding what that bar of excellence needs to be for first MVP and getting those signals really quickly, that's what makes some of these things difficult.
Randy Silver 23:39
When I talk to people at, you know, Google or Metto or OpenAI, places like that, you know, the interesting part is for them, tokens are almost free, at least in the development side. But now you're balancing, you're running, making simulations in real time at scale. That's not free. That's that's going to generate a cost. So at what point are you balancing the cost of providing this experience versus anything else? You know, there are plenty of people who use customer service bots in early days to generate code because they were jailbreaking the LLM in there. Where is the balance between something that is a free experience and good enough?
Soufi Esmaeilzadeh 24:19
Yeah. I think uh actually, Randy, all the time that is a question we should be asking ourselves because the same I was telling you in this new AI world, people can build really quickly and prototype really quickly and demo really quickly. It's fantastic. The other flip side of that is you can have a ton of ideas and a ton of things you can do. And I think as leaders and in conjunction with our team, the question is is it worth it? Is this thing worth it versus these five other opportunities that we have? Whether it's the cost of people, attention, uh, tokens. And so that's the stress of in a world with AI where you can build really quickly. The flip responsibility I think product managers have is build what really matters and that makes a difference. And I think associated to that, build it well because it's really exciting to get something out really quickly. But it's that kind of craft and excellence that like consumers see it. And when you don't do it right, they will tell you you did not do this right and they will stop using it. Um and that's the bar we have to continue to maintain and the and the culture we have to maintain in a world where everyone is moving really fast and has a ton of ideas and they're really excited. Um, so that's how I really think about that trade-off.
Randy Silver 25:30
The pace of innovation is really relentless, the expectations are relentless, at least for prototyping. And then there's there's prototyping and then there's getting stuff into production. And we can talk about that challenge in in a couple of minutes. But let's talk about the team that you've got because I'm seeing lots of challenges about how teams are structured these days. And how do you approach that with your team? What does your team look like in terms of disposition? What is the nature of a product team? We for a long time it was two pizza teams, and now people are going down to happy meal teams in some cases of just one person. What does a team look like for you? What is good? How do you then ensure that you're getting quality out of your people?
Soufi Esmaeilzadeh 26:12
Yeah. I um uh for one thing, I think I have the benefit of really being able to think about and craft my organization from the ground up in this AI era. And the term that I like to use, uh, pizza french fries is um SWAT team structure, really. I it's a lean, flat team. I look for pods where I have one uh senior lead who I look to be really like a strategic thinker that can ensure across the board we are making the right bets and thinking about things the right way with a very strong bench of ICs, small bench of ICs. Um, and the reason I use the terminology SWAT team is I don't think, first of all, these days, you need a lot to do a lot. I don't and well, you don't need a lot of people to do a lot, you need the right set of people. The second, the SWAT team analogy, because I want to keep the lines of communication very flat between myself and the rest of the organization. I do not want multiple layers. When you have multiple layers, you don't have context. When you don't have context, it's really hard to move fast and ship and ship with excellence. Like you need those lines of communications to be there. If anything, just to enable your team and help them be unblocked and make sure that they have the support that they need. The second thing I think about, and I think about this not within just my own team, but more broadly, is really, and I talk a lot about this a lot, building horizontally. And what I mean by that is, you know, in a world where you have these very capable models, you really want to build at scale. I often say you want to be inspired by vertical use cases. You really want to put yourself in the consumer shoes and think about a journey that really matters to them. But you really want to try to challenge yourself and say, can I build this horizontally? And on a team basis, the effect of that is we are working more and more in these like virtual teams. Bring in the right talent from different departments into it, but you're really breaking what may have been traditional organizational silos because you're building uh differently to enable good people to do good work.
Randy Silver 28:21
One of the things I keep seeing is organizations that had bad practices or any of their bad practices, even if they were largely good, they're getting hugely magnified by the pace uh the pace and the tools. Uh, but you know, the good practices are going even better, but the bad practices are getting in the way way more. How have you ensured that you're uh doing this at a the right pace and getting the right quality? What what have you done with the teams to ensure that uh as the pace increases and everything is changing, that some of the basics are really addressed?
Soufi Esmaeilzadeh 28:58
So a couple things, and I hope I am doing this every day, Randy. One is leading by example and setting the right tone. So that bar of being the user, thinking about the underlying technology, challenging ourselves, just all showing up that way. Because if I do it, the team does it, and that's critical. The second is I think while we have a view of the destination, like we have an opinion of what search is going to become in the future that we're building and how we are unique because we are search and we have web content and we want to organize it. And there's a very clear line of sight of where we want to go. I really do work with the team on what are we gonna do, however, in the next uh three weeks, three months, six months, and having alignment and that clarity there on the speed of execution that we want to maintain. But the third that uh I spent a lot of time talking to my team about is you are the premier consumer AI team. And what that means is you represent the user and you represent excellent. You represent craft, you represent quality. If you build that way, if you really dig in, if you understand what's going on with the user, then we can ensure we're shipping high-quality product, even if we are moving fast. And so those are kind of the things that come in every day in the way we work and the way I work with the PM team, and the way to then work with their counterparts across the board to ensure it. Now, the other things are traditional product is that you do not let go of. You have clear hypotheses on why you are doing what you're doing. If you do not have that, you are not building the right thing. You have clear goals with your counterparts that you're all bought into. You really understand what the destination is, what the metrics are that guide you. Um you spend time early on together defining what good is. Show me what that product is that you think is good, and show me what not good is. And is everyone speaking the same language? And those are just exercises that I think are good practice. It's good foundational product work that uh was always necessary that we should continue to bring, even in this environment of moving fast.
Randy Silver 31:07
Have you had to change your processes around the pace of change, or is it always been very quick for you?
Soufi Esmaeilzadeh 31:14
So I, you know, it's a good question. I think that um without a doubt, the speed of execution has increased in recent times. Uh, you know, I think we are shipping much faster. Uh, we are innovating much quicker. Expectation of speed of development that I have for myself that I bring to the team is much higher. I think that there is a sense of urgency to meet this moment all the time. And what's interesting, and this is what's been made me really happy, is when I look at how people are showing up, they're excited by the sense of urgency. People are excited that we can do so much that in the past either wasn't possible or would be such a huge lift to get done. Uh, but absolutely we should all acknowledge there is this sense of urgency and responsibility that is more acute given uh the AI shift that's happening.
Randy Silver 32:11
Has this speed uh been applied consistently as in getting to prototype is one thing, and then getting to to general availability, getting it to everyone, that's another thing entirely. So, where where are you measuring this? Are you measuring it at getting it out into beta and getting it into the first stage and then potentially going through more traditional stage gates or what's what's the approach and has that changed?
Soufi Esmaeilzadeh 32:35
I measure it at every stage. Every uh really, because I think uh when I say speed of execution, it really is at the end of the day, what I care about is is it helping people? And then did I get the signals? And then am I continuously making it better? So that idea of constantly moving fast with a high bar and it's hard. Do it, do it fast and do it well. Um, but most importantly, do it well. Do not, I do not want to put out products there that do not meet the bar of excellence, the bar of trust, the bar of quality, the bar of craft that people come to expect a search. It is at every stage of development. You know, when I meet with my team, again, we're very clear on what, where we want to go, but also I really bring it home to where do you want to be in three months from now? It doesn't matter what stage of project you're working on. We have folks working on very novel, kind of frontier ideas, and we have folks that are working on more established uh experiences that are out there. It's still that question of where do you want to be three months from now. Um, and I think that's the right way to think about it. Uh, because like I said, the technology is moving fast, the uh consumer expectations are evolving really quickly, and and so we need to be showing up with that same sense of urgency.
Randy Silver 33:47
With everything moving so fast, one of the challenges I keep hearing about, and and to be honest, seeing myself, is the challenge of uh context switching. You know, you've got so much going on. So, how how are you and your team avoiding problems related to that? How are you in the team avoiding burnout from just trying to keep on top of everything at this relentless space?
Soufi Esmaeilzadeh 34:07
If I told you coffee, would you believe me?
Randy Silver 34:11
No, I'm in England, it's tea here.
Soufi Esmaeilzadeh 34:14
Many, many cups of coffee. Um so uh this is where I I think my job as a leader, one of the critical things is to give people sort of like very clean direction and minimize the noise, first of all. To your point in a world where there's a lot happening, you can very quickly see a lot of noise coming in. And that's one of the worst ways, in my opinion, to work. It's okay to work urgently, it's okay to develop quickly. Chaos, spinning, debate, uh, that is uh the thing that I try really hard to spot in the system and clear out for folks so that they can do their best work. Um I have not seen uh the effect of like my team does not go, uh, this is moving too fast, I don't like it. They're like, this is actually energetic. I get to do great work. But I think the same is true that if your team is trying to do great work and they're spending their time uh sort of like debating with some other part of the organization about whether we should do something or not, or they find themselves not having resources because XYZ is not convinced, or they're saying, hey, is this other part of the system doing what I'm doing? That's the kind of energy. That's the kind of noise that I think as leaders we need to eliminate for people so that they can bring their best work forward. And then the results for me so far of what I've seen is people are energized, they're excited. It's not that they don't want to work at this pace. So that's what I that's what I really uh look out for. And listen, at the end of the day, I talk about urgency and energy. We have to build sustainably. Okay. I I think I mentioned I have two kids, they matter, I need to pay attention to them. So you really have to build sustainably, which is where you got to be really, like I said, focused on what's your destination, what are your priorities, what matters, what can wait, and then help your team think through that too.
Randy Silver 36:10
So okay, I think we're running out of time. So I'm gonna ask you a couple of questions if about your team, if that's okay. And I'm just curious about as you're hiring now, as things have changed and and the pace and the the skill set and the disposition of team has changed. What do you look for when you're hiring and promoting people? What's the what are the key things now?
Soufi Esmaeilzadeh 36:31
So uh first of all, for my organization in particular, I talk about listen, I'm really looking for consumer AI PM DNA. It will qualify this. Given the fact that we're driving the AI experience on search, the consumer side of the definition, like I need people who have demonstrated that they can really build user-facing products, they can be the user, they can put themselves in their shoes, uh, they can really uh be creative about thinking what are the kinds of products that could resonate. So that's number one. The AI component of it is for me, I'm looking for people who are really keeping up to date with the underlying technology, where it's going, what opportunity it's opening, what other products are emerging out there, constantly maintaining an eye of the landscape because it helps them come and say, I actually think we can either A, build this new capability and here's why it would matter, or B, build differently. The same problem space, but let's approach it differently because the technology is enabling us to do so. And then third, this uh zero-to-one DNA where I can find it, I really gravitate towards it. People who have this builder startup mentality, this excitement of getting things done really quickly, of making it happen, of pushing the boundaries and moving at speed is something that I really seek in the DNA of the of the PMs that come and join the team.
Randy Silver 37:53
A couple of months ago, the the thing everyone was talking about was evals. And the last couple of weeks, it's been, it seems to be harnesses and things like that. So um, in terms of the specific skill sets, you know, the real challenge that I've seen is taking a step back and realizing that the LLM, the agents that you're working with, they're a bit obsequious in a lot of a lot of the time. They'll tell you what you want to hear, they don't push back very well, they will absolutely uh um accelerate what you're doing, both the good and the bad. And if you it's a lot of rigor of your own practice to do it well and put the guard guardrails in place. Are there any key skills that you work on with the team or that you're saying or that you'd advise people, this is the thing that if you double down on this, it really pays off over the next few months?
Soufi Esmaeilzadeh 38:44
So I think that where, for example, we're spending a lot of time having conversations is where is it that these tools are helping us accelerate the way we work? However, with that, as a team, we often have aware of the limitations of them. Um and what we have found is they're very useful in getting a lot off the ground, whether it's like you want to do logs analysis and now you can actually do it in a much easier way, or you want to build a prototype and you can get your hands out there. Uh prototype perhaps less of a better example of the thing speaking back to you, but whether it's you want to sort of brainstorm an idea, uh, it is great at this stage for like getting things going, but but we still recognize that we have to challenge and cross-check and verify, and that's okay. There's a lot of value in lifting that load and helping structure the thinking, for example, but also a lot of a discussion on the teams. Where have you seen the limits? How are you then uh bringing that into your workflow to ensure that you stress test them? And I'm sure those limits are going to change in the next few months. And so keeping that conversation in our team organization when we meet, when we have our group meetings and we're sharing ideas of like, well, how did you go about this? And how did you go about that? And that's where we can also have an honest conversation of what worked, what didn't work, where is this really adding value in your productivity? Where is it less so? So that's a constant conversation that's being had. And like I said, when I look at how we built a year ago and how we're building, how we built six months ago, and how we're building today, and I am positive, I joke about how uh my job feels like it's changing every six months. We are gonna build very differently three to six months from now. Um, that kind of knowledge sharing is integral in an organization right now because everyone is coaching everybody.
Randy Silver 40:27
And how are you thinking about longer-term resource allocation? In that it used to be that you would have teams organized around certain customer segments or feature segments or parts of a user journey, things like that. And you could plan, I mean, I hate to say Gantt art, but to a certain degree, you'd say, okay, they're be doing this, then they're gonna roll off the they'll they'll have launched this feature, they'll have accomplished this goal, and then they can move on to the next thing, or we can trade some people around. But now you're having to deal with model drift, you're having to deal with constant maintenance of something and really you own something and you have to continue to look after the existing experience as well as the evolving experience. Does that change the way you think about longer-term team disposition, or has that come up yet?
Soufi Esmaeilzadeh 41:15
So I think um I think the question that you're asking is with this constant shift that's happening, how do you build sustainably an organization around it? Um and I think for me, it's actually defining areas. The areas will stay the same. So let's say I have an area that's really accountable for the end-to-end response that the user sees. Uh, let's say I have a person who's really an area that's really responsible around just the overarching sort of like uh consumer product, the framework of it. Let's say I have an area that's really responsible around like long-running journeys. The areas for now are at least sustainable enough for a period of visibility that's not like three months. It's sustainable. I actually think these are areas that we're gonna invest in. Then within that area, it's okay that the way you build and solve the problem is varying. The model technology is varying, the technical approach is varying. Uh again, the challenges are evolving. That's okay, but at least your area is somewhat sustainable because then you can formulate an opinion of where you're going and what good is. I want some amount of longevity. Now, I can't tell you if that longevity is two years, three years, but it's absolutely longer than six months, nine months, because I want people to think about the problem space, have accountability for it, go deep in what they think success looks like. Uh and so that kind of continuity really matters to me as I do the org design. And as I then uh related to that, organize the SWAT teams that I want. Because those SWAT teams build muscles and expertise in each of their areas.
Randy Silver 42:52
Sufi, the time has flown by. We've gone to the top of the hour. I'm sure you've got 10 other meetings you're supposed to be at. So thank you again. This was fantastic.
Soufi Esmaeilzadeh 43:01
Thank you so much for having me. I I really enjoyed the conversation today.
Randy Silver 43:05
The product experience hosts are me, Lily Smith, host by night and chief product officer by day. And me, Randy Silver, also host by night. And I spend my days working with product and leadership teams, helping their teams to do amazing work.
Soufi Esmaeilzadeh 43:21
Lure on Pratt is our producer, and Luke Smith is our editor.
