Francois Lopitaux is SVP of Product at ThoughtSpot, the analytics company built on the premise that you should be able to search your data like you search Google. He began his product career 18 years ago, when Salesforce acquired his Paris company, and went on to lead customer service, analytics and AI products there before joining two startups and then ThoughtSpot. He joins Randy Silver to explain how building agentic products differs from shipping deterministic software, from designing without control of the user path to keeping an LLM trustworthy when the answer has to be the same every time.
Key takeaways
- AI has cut the time to prototype and validate, and ThoughtSpot estimates its development is two to three times faster. Productionisation still needs sound architecture, guardrails and experienced engineers.
- When everyone can vibe code, discipline matters more. Lepiteau compares fast AI-assisted development to driving a Formula One car on a road built for a mini Austin: the speed is only useful if you can control it.
- Agentic features are probabilistic, so product teams are now building guardrails around an LLM rather than a fixed step-by-step flow. That means more hypotheses and more testing.
- Keep the end goal fixed and stay flexible on the technology. ThoughtSpot is now revisiting existing features to see where newly released models can replace what it built with LLMs.
- The eval system is becoming the new PRD. It defines what good looks like and what must be avoided, in a product where the UI is a prompt and the path can't be predicted.
- For analytics, where "what is my revenue this year?" must always return the same answer, trust comes from using an LLM only where it is needed and supplying a semantic layer. That layer defines how your business works, so the model behaves less like a new intern guessing at your terminology.
- Adoption and repeat usage are the metric that matters most, backed by anonymised sampling of answer quality and follow-up questions. Enterprise customer interviews supply the qualitative temperature check.
- Embedded and white-label products raise the bar on trust, because customers ship the product to their own customers and carry the accountability. Features must be rebrandable and opt-in, with flags so each customer can roll out at its own pace.
- An AI feature is never finished. Evals are only a guess at how people will use the product, so teams must keep observing real usage after general availability. ThoughtSpot organises teams by product track, which makes that follow-up a natural continuation of the work rather than extra load.
- Pruning matters more now that code is cheap to generate. Lepiteau's own team has built three versions of its conversational agent, and without cutting back old versions he warns of a monster where every addition breaks something else.
- Lepiteau still hires technical, curious product managers. For a recent associate PM role, he asked candidates for a Git repo and a video explaining why what they built solves a problem.
- Execution is speeding up, so he expects smaller teams and a falling PM-to-developer ratio. Deciding what is meaningful to customers is the remaining bottleneck.
- 2025 was about proving AI works. Lepiteau expects 2026 and 2027 to be about proving ROI, so teams should be able to switch models easily and pick the cheapest one that does the job.
Chapters
00:00 Introduction
00:48 Meet François and ThoughtSpot
02:55 How AI is changing the product development lifecycle
05:18 Why discipline matters more in the AI era
08:00 Building probabilistic features
09:43 Why evals are the new PRD
13:57 Building an eval system for agentic analytics
17:15 Creating trust with a semantic layer
19:30 Measuring the success of AI products
22:12 Trust in embedded analytics
24:19 Go-to-market for white-label and OEM products
26:54 Rolling out at your customers' pace
28:37 Why AI features are never finished
31:20 Organising teams by product track
33:42 Pruning and sunsetting features
35:38 What to look for when hiring product managers
37:56 The future of team size
40:41 Managing AI costs and model flexibility
42:49 Wrap-up
Episode transcript
Francois Lopitaux 0:00
First and foremost is the numbers, right? How many people are using your product? How is the adoptions? How many people like go back to use your product? And this is for us is an amazing matrix. And for me, is pretty much the only one but matter, right?
Randy Silver 0:21
Hey, welcome back to the product experience. I'm Randy Silver, and today we are talking about building a genetic system, how the product lifecycle has changed. It's the thing that I think is on everyone's mind. It comes up in every conversation I have with product people. And we went and got someone who actually knows what he's talking about for this. So, Francois, do you mind doing a quick introduction and sparing me from mangling your last name?
Francois Lopitaux 0:48
Yeah, sure. So my name is François Lepiteau. I'm SVT of Product at Sotspot. And uh I've been uh in a product for quite some time now. Uh I had uh I had the pleasure to start to be a product manager at Salesforce about 18 years ago. So my company in Paris has been acquired by Salesforce. I had pleasure to move to San Francisco, and since then I just you know love the job. Uh I work at Salesforce, so big quite big companies on uh on customer service side, so building like customer service applications, then more like on the analytics and AI side. Then I joined two small uh startups, and uh since about two years now I'm uh working at SouthSpot.
Randy Silver 1:30
Fantastic. So tell us a little bit about ThoughtSpot before we go any further. Just what's the background? What is what is ThoughtSpot?
Francois Lopitaux 1:36
Sure. I mean, SouthSpot, you know, is a company that's been created about 10 years ago. Uh and the premise was you should be able to search your data like you search on Google. So it was really like the core technology was a search-based uh analytics product. And obviously, uh with the raise of LLM, it's enabled us to go much further than that and really like make it self-service, make it like uh really easy for people from anybody, any profiles to find their data, find inside, take action now with a combination of the search index and the LM technology.
Randy Silver 2:10
So you're doing this natively, and there's a lot of things you know, the moving from the traditional software development lifecycle to an AI-enabled one, things are changing radically for lots of people. And I'm seeing huge differences in the development process with the people I work with and also everyone I talk to, but we're also running into some of the same problems that are coming up again and again. You know, we're able to prototype faster, but getting to production is still hard. We can generate code faster, but code reviews are still a huge gateway. What are the problems that you're seeing come up? What's what's changing in the development lifecycle for I think you know the the the first point is really the opportunity, right?
Francois Lopitaux 2:55
What is bringing uh and what is the value we get from this uh this AI world? The first one obviously is the time to prototype, the time to do testing, the time to validate hypotheses is got reduced a lot. So now uh a product manager can uh kind of like you know generate a prototype with a rich code. Even myself, for example, I have generated uh two applications for the company. I I had a background of developers, but it was like 20 years ago. I was a bad developer, so that's why I became a product manager because I was, you know, I was to myself, I told myself like I will never make a good career on the development side. But the reality is because of this AI, I'm able to generate things, right? I'm able to generate things that either can be become a product, if they are kind of simple in terms of architectures and complexity, or I can build like very complex prototype in a way. And I think this is really a game changer for product manager. Now, you know, is are we are we going faster? Yes, we are going faster, but I don't think like if you really want to have a strong product, you still need to spend time on productionization, right? You need to be able to create a strong architectures that is going to be sustained in the time, that have the right barrail, the right trust. And for that, you know, the architects, the developers are still like uh much well uh welcome and they have a strong hold to stand there. And I and I think also that uh actually organization and discipline become even more important because that now everybody can vibe code, everybody can generate things. You need to have a discipline to be able to decide what is going in, what is not going in, what do need work, what don't need to do have work. So I think it's it's even like you know, it's the AI is creating a little bit of chaos. And if you are not organized and if you have not disciplined, then the chaos is going to kill your company. So I think it's super important to have this discipline, even more important with the AI world, which is kind of contradictive because you may think like because of AI, I can do everything, you know, product become builders, everybody can ship code and everything like that. But I think if you really like to resist at the maximum, right now with a level of the technology, you are, you know, it's a recipe of disaster.
Randy Silver 5:18
Yeah, definitely seeing that, you know, we've been trying for years to let teams be as autonomous as possible. And that's created absolutely created chaos because everyone wants to work in a slightly different way. And what we really need is a minimum viable bureaucracy where people work to some basic standards. And I was talking to a friend who's at an enterprise today, and as they're migrating their teams onto more of an AI uh development lifecycle, what he said he's seeing is the teams are uh adopting and they're being forced to adopt to a standard. And it may not be something that's changing the way that they work on a day-to-day basis, but the development lifecycle that underpins what they do is becoming standardized. So are we actually enforcing better discipline through the back door by by going migrating this way?
Francois Lopitaux 6:07
I think we are definitely removing, you know, like code, you know, you were speaking code review. Code review now are done by agent, right? Simple bugs can be fixed by agent. So there is definitely accelerations and and it's going to go even further out with the improvement of the LLM. But I think the discipline is super, super important to be able to create a consistency, to be able to create something that is not going to be uh demo well on day one, but is also going to be sensible for the future, you know, because your product, you are going to add more and more features, and your foundation needs to be super strong. And you cannot be like you're just things, you know, added to each other that become some kind of like globes that nobody really knows how it's working. So I think that uh you know discipline definitely is is super important. I it was before, but because before, right, everything was going more slowly. So you had time to kind of catch up before it was it was going bad. Now, because everything goes faster, you need to be able to control this fast speed, you know, Formula One that is driving on the road. Before you had like your your small uh car, slow car, like your mini Austin, it was okay. You had you had time to see it coming. Now you don't see it coming, so you need to put everything in place to be able to catch it.
Randy Silver 7:26
Is everything actually going faster? Because I definitely see the getting to prototype stage, you know, that first 80% of figuring out what is it that we want to do, being able to describe it to people well, getting to not a PRD anymore, but a prototype so that we can work from that and work uh communicate better together. But that second 80% of the pro of the project, actually getting something robust, scalable, reliable to production and doing it consistently, that's still really hard a lot of the time. Or are you seeing it work differently?
Francois Lopitaux 8:00
I mean, we see it going faster. Like we think like, you know, we go at between two time and three times faster, uh, for sure, in our development. I think the the tricky part is is the fact that the way that we are building features is different now with AI, obviously, with code and stuff like that. But also the type of feature that you are building is different because now you know you are not building any more like predictive features. You know, before it was like, okay, I'm inside uh customer service applications, uh, I'm going to create a flow to create a case. And this case is going through different processes. And then when the case is closed, it's going to mark it as done, and I'm going to move on to the next case, right? So the feature building was much more easy to do because it was completely predictive, right?
Randy Silver 8:51
Right.
Francois Lopitaux 8:52
Now, at the same time, you have to think about not anymore, because you have to take into account the probabilistic nature of LLM. And the fact that your feature is not anymore like step one, step two, step three, and I'm done. It's going to be basically like creating guardrail to the probabilistic nature of LLM. And it's much more complex because first you don't know which feature is going to work or not. Right. It's all about hypotheses. Uh, you think that this is going to be great or not? So you have to do much more testing. And after you don't really know like if it's the best way to address this problematic, right? You have a problem you need to solve. I am taking the best road to solve it. And so that's also like I think the the two components changing at the same time is kind of like interesting in the in the industry right now.
Randy Silver 9:43
So let's let's ground this in in how you're actually working right now. Because, as you said, you know, we used to work in a very deterministic way. Uh, we have a hypothesis, we know what the customer problem is, we want to solve it. We're going to draw up some screens, we're going to draw up a path, and we're going to say, does the cut can the customer get from A to B to C to D in a realistic way? Do it quickly, not too much friction, not too many, uh, not send too many people to support calls, things like that. Now it's we want them to solve it. There, we've given them a chat interface in many cases, and we can't control how they're going to get from A to anywhere. We have no idea what the where they're going to go next. So, how are you even starting the process of designing the feature and and figuring out just walk us through what it's like for you to develop a product or develop a feature these days?
Francois Lopitaux 10:36
Yeah, sure. I mean, I think that you need to be first, you need to be very strong about what is your target, what is the end goal. And you should not be too much attached to how you achieve this goal, what technology you are going to use. If you are going to use NLM or something else, and you know, like if you connect to the news, right? So the JEV models that just uh got on the news last week uh is even changing the game, right? Now you are going to think about replacing some stuff that you were doing with LLM with uh with a JEV, and then it's going to, you know, all your classification questions that you used to use LLM is going to be done through this new process. So again, I think it's really important to think about the end goal and the mean may change a lot, uh, like we see now, right? Now we are at the company level, right? We are now looking at all our different features that we have. How can we optimize with this latest technology that has been uh out on the news, which is super interesting. The second thing is, and this is where it becomes super important, is the Aval system. Because your Aval system is almost becoming new PRD. The new PRD is an eval system. Basically, you want to validate what is the behavior you are looking for. Because again, uh you don't control the UI because it's prompt-based, right? As you as you mentioned, it's conversation. So you are, but you know what is good means, right? You know what looks like what you want to achieve, what goods mean. You know also what exactly you don't want to uh accept, what you want to avoid. And uh and that's why you know the new eval is really a way to control a little bit more what you are looking for. And I think the eval is obviously like super important, not super easy to create all the time, but it's really important. And I think the last point which is really important is, and it was true before, right? When we were saying like when you ship a feature, it's GA done, you move to the next features. A good PM obviously don't take like that, and they continue after the feature is getting G, and you know, continue to work with customers to be sure that the mission is fulfilled and and they like what they have built. But I think with AI is even more, it's uh you know, it's increased because you don't even imagine what people are going to do with your technology, and you need to be able to observe, you need to be able to continuously like check what people are doing with it to be able to adapt super fast, super quickly, and change the behavior that you are seeing. So that's I think the the the new way to build stuff.
Randy Silver 13:15
Okay, so you you mentioned a few different phases there. So let's let's walk through them a little bit. Again, a more traditional way of designing apps, products, experiences, things like that started with, you know, we looked at the data and the content, but we looked at the way that people interacted through with it through a UI, a static experience that we designed. And now we've got something that's completely out of our control in many cases, uh, where the data and the content often is the interface. So making sure you've got a quality layer there before you even start is really critical. How are you scoring and evaluating? Do you have what you need to provide a good experience in the first place?
Francois Lopitaux 13:57
I mean, we spend a lot of time to build our Eval system uh because it's critical, right? You have so many parameters that you can change in your system that is going to impact a lot the result. So we spend a lot of time to look at Eval. What is good things for us is we have also public benchmarks that are available. So, for example, for our technology, you know, because we are an adventic analytics product, we are um things like birds that we can use or spiders that we can use, where we can run through a typical question, what is the out, you know, what is the expected outcome, be able to validate each other in terms of consistency, in terms of like pertinence, accuracy, and so on. And so we are just you know using all the time, every time we change something, we have to run this eval because there is new LLM models going out, because we are building more features, and one of the you know biggest features that we invest a lot is the context, right? Because of the nature of the LLM, which is purely probabilistic, and because analytics per definition need to be deterministic, right? When you ask one question, you all the time want the same answer. And the question are, you know, I love LLM for the creativity, but when you ask, like, what is my revenue this year, you don't want LLM to be creative, right? You want to be like deterministic. And so the the big stuff that we have created on top of Eval is to be sure the time is to create the right context and as much as possible context that the LLM is kind of like you know going to the right direction most of the time. And also, obviously, is to use LLM when we need to use LLM. And I think this is really important is really pick the right technology at the right point to have the right outcome. If your outcome is purely probabilistic, then you don't really care. You want the most creative answer, you should do, you know, you should do LLM everywhere. If your answer needs to be deterministic, then you should be careful because every time you add LLM at a specific point in your algorithm or in your features, then you are creating some divergence. And then you have to create something against it to be able to control it uh as much as you can to be able to really like arrive to the right uh results. So I think it's you know it's super interesting to be able to play all these different technology, uh, but the AVA is staying like the reference if you want to be sure what you are building.
Randy Silver 16:24
So, how are you doing this for analytics? Because you know, there's the all these all these lines about lies, damn lies, and statistics and liars figure and figures lie and all the all these things. Um we've always been able to be creative with the the numbers that we've got available to us. We can always interpret them in different ways. And especially when you're working with something that, and I don't want to anthropomorphize, but it's trying to be helpful, it's trying to tell you what you you what you want to hear up to a point. And if you if you've got everyone in a company asking questions uh of something that it may start in the same place, but the way they interpret it, the way they use it may be very, very different. How do you keep people aligned and make sure that the stuff is reliable, that it's accurate, that it's actually useful uh rather than just nice?
Francois Lopitaux 17:15
Yeah. And I think you know, this is what is interesting with Olai product is you, you know, everybody can build a demo, right? And the demo will look amazing, but you this is not how you make it your product successful. You make your product successful with people using your product because if they use it, it means like they will trust it. And trust is like the cornerstone of everything. And to get the trust, because you are using LM, you need to counterbalance that. And the best way to counterbalance that is first to use it only when you need LLM, and secondly, is to really like work on what we call the context, the semantic layer in our space. So be able to describe as much as possible how is your business working. Uh, you know, when you describe what is what mean a new customers, how you define a new customers, how you define revenues. Because your LLM is like a uh a new intern, right? It's amazing, you have amazing potential. But if you don't know how you call your business, how your business is working, what is your definition uh in your company, then it's going to make poor judgment and poor analysis. So once you have this defined, this context ontology defined in your system, then this is where your system is going to create trust because people are going to be able to understand that it's doing the right things. But also, this is where you will have a consistency among your users, where everybody asking the same question will get the same answer because, again, it will be grounded in your company uh knowledge, in a way, on this old context and semantic layer that's, you know, for example, in our case, we have built into our system. And that's why our system is super accurate and also where people can trust it because of this layer.
Randy Silver 19:01
And how do you measure success of this? So now, you know, again, we used to use very traditional analytics of how many times did people come to the page, how many times did they click on this button, how many times did they use this feature? But now you're trying to do something that is both quant and qual in this. You're looking at are they happy with the experience more and potentially more than how much did they use it? Or what is the the way that you're evaluating uh how successful you are with these products?
Francois Lopitaux 19:30
Yeah, I mean, there is multiple ways. First, you know, first and foremost is the numbers, right? How many people are using your product, how is it adoptions, how many people like go back to use your product? And uh, and this is for us is is is an amazing matrix. And for me, is pretty much that's the only one that matters, right? If your product is successful, it means like people see values and they are going to use it more, and it's a validation of the of what you are doing. Now, obviously, we are doing other things. What we are doing, for example, is we are sampling, uh, we are anonymizing, obviously, like uh the different adoption metrics in our platform. But after this anonymization, we are driving metric. Like, how much this answer looks good, how much this answer looks to answer the questions. Uh, is there any follow-up question to your previous answer? So have some kind of like numbers about your overall usage of your product to be sure that you are going the right direction. And every time you are pushing a new feature, you are pushing a new feature that is improving your system. So it's really like if you summarize it, that's two levels, right? Level one is having values from customers and customer validation, and value two is on our side, we are continuously monitoring our system actually, to be sure that uh we are delivering values to these customers. And we can also react very fast if we see like early warning system.
Randy Silver 20:56
Quick one for the product folks listening. AI can draft the PRD, prototype the idea, and help you build almost anything. But knowing what to build, that's the hard part. Jira Product Discovery helps product teams bring customer insights and ideas together, prioritize what's worth building as a team, and create living roadmaps that connect directly to delivery in Jira. So when more ideas are possible than ever, your team can focus on the ones that matter most. Try Jira Product Discovery for free today at atlassian.com slash MTP. Yeah, I always find this stuff really interesting because the quality of a conversation is always an interesting one. You can have a short conversation because you got to the answer that you needed really quickly, and that's value, but it doesn't show a lot of usage per session versus you can uh mandate that people have processes in a company that inflates the number of sessions, but it may not be the the actual value that people are getting. Lots of things like that. So yeah, it's always I always Find this stuff really interesting about the right way of focusing and making sure that you've got that early warning system in place. I guess things tailing off is the biggest siren or the biggest warning.
Francois Lopitaux 22:12
And I think also, you know, we are enterprise customers, right? So we spend a lot of time with our customers also in more like interviews, feedback loop, conversation, which also like is more like you know, uh qualitative, but it's also very interesting to feel the temperature of our customers. I think there is also like something that's super important for us, is because half of our business is really like analytics for employees, but also we are powering a lot of companies that don't want to rebuild commercial analytics, uh, but they still have a product where they have data they want to share with their own customers, right? So we are providing what we call embedded analytics. And this is becoming even more important because here, customers who buy our product, they are going to ship it to their own customers. So for their own customers, they become responsible for it. And so it's uh, you know, it's an even bigger trust than it to be able to achieve, because they are going to take the accountability about the service that you are providing. Which means obviously, like it's even more complex. And the trust is it's much harder even to get their trust because they want to be sure before they roll out something to their own customers, not going to you know, uh do bad behavior, because it will, at the end of the day, impact their brand on top of our brand, obviously, but it will impact also their brand, which uh which is uh really important for us. So that's why the trust aspects is super important, especially in our case with this kind of like second level of customers of customers.
Randy Silver 23:46
So that segues really nicely into where I wanted to go next, which is the next step of the product development lifecycle. Um go to market. You so you, as you said, you're essentially B2B to C. It's beyond white labeling, it's disappearing into a customer, uh into the client product. And it is part of an overall experience. So you can't say, hey, we've turned on this new button, this new feature, this new page. It's we have a new capability baked into the interface that you've got. How do you handle go-to-market on something like that?
Francois Lopitaux 24:19
I think it's you know, it's also first is like from the product event development, it's complex, right? Because from the code product development, everything you build something, obviously you want to brand it in your mark, in your brand, right? So we have spotter, which is our agent X solutions, and of course we call it Spotter everywhere. And so when the our agent answers say, hey, Spotter or Agent Spot is our technology. When you do white labeling or EM, they want to bring their brand, right? So which means like every feature that you build, they need to be able to make it their own features. So you need to think all the time that when you kind of make it your own, you need to make it possible to make it their own at the same time. And also it means like sometimes you don't want to push too much on the on the old branding because you want to keep it like kind of like uh high level. So that's that's quite interesting. The other thing which is kind of complex also is every time you push new features, and you know, these days uh it's going, as I told you like at the beginning, much faster for us. We we've been much more capabilities. Every time you need to let people uh the time to adopt these features, right? Because they have to be able to control the rollout of every capability. They need to be able to potentially train their customers to use the new capabilities. So we need to also be careful when you are pushing new stuff uh to make it like opt-in base in a way, or to be able to uh make it seamless or design it super well that you know you can avoid training or things like that. But it's just like adding another layer of complexity when you are building uh for others, when you do the white labeling and the OEMing, uh, because this is all the components you need to take care of. And even if you are super excited about new features and you are pushing to everybody, some customers may say, like, oh no, I don't want that. You know, I I want to control the rollout, I want to control the experience. Let me time to do that at my own pace. So it's just like a lot of complexity added to what we are doing.
Randy Silver 26:27
That's one of the points that came up in the the when we were drafting the makers manifesto was that specifically is we have the ability to move at speeds that we never could before, but that's no good if our customers uh and our partners can't adopt the features that we're doing. We all can only move at the speed that they can consume it and find value from it. So, how are you how are you measuring that? How are you making sure that you're aligned with them on that?
Francois Lopitaux 26:54
You know, we we obviously spoke uh to a lot of them, but the the complexity also is we have a very different type of customers. We have customers which are like startups who are moving fast and we want to have the latest all the time. And we have more traditional vendors where you know their customers, for example, are not as savvy uh as the startup world. And so you need to be able to adapt to the different pace of people. So, you know, very tactically speaking, what we are doing is there is a lot of flags in every direction to let people to really like decide when they want to turn on this or these features because you need to be able to go at the speed of your customers, go fast with the one who want to go fast, and go at more reasonable pace for the one where you don't want to go as fast. So you have to adapt and be flexible.
Randy Silver 27:53
The next part of the development lifecycle post-launch, you know, we're used to you launch a feature and you do a bit of a shakedown, but after that, you're pretty much only looking at it from a bug fix perspective. If we need to do an upgrade, if we need to do a migration of stamp at some point. But one of the things that comes up with this is models drift, they need to be tuned, they need to be uh, you know, you need to pay attention to them over time. And that fundamentally changes potentially how you and your teams are are being staffed on this, because how much of your time do you need to uh relegate to reserve to make sure that you're constantly keeping the the models up to date and and doing the maintenance on the on products that are released?
Francois Lopitaux 28:37
Yeah, and I think that's you know, it's definitely like before when you were created a button on the page. When the button is created, you move on, you go to the next things. Now your feature is never finished. You never know if your features is really going to work until people are using it and providing you feedback. So you really need to spend a lot of time on the after once you have shipped, uh, to be able to see with customers like, is it working as expected? Do they get the values? Because honestly, you may you may create the best eval. Eval are still like you know, a guess that you are creating on your side about how people are going to use your product. But like people are obviously very different ideas that you may have. So it's it's require a lot of time to you know continue after after the shipping. And uh and it's kind of also interesting, uh, but it's you know, it's obviously like taking a lot of time, and uh, but it's great interaction. I think is also you know, like what we're seeing today is like, hey, you know, everybody can do everything, right? A product can do it with developers, a developer can do a product work, a designer can do a product and and build a product. But the reality is also people have some affinity, right? If some developers, they really want to build features and they may not want to interact too many times with the customers. Some product guys, and they want to be features, but they really like to interact with people. And I think that you know this is also where the role is really important because product managers, by definition, are customer-facing, right? By definition, they need to interact. So with this new type of AI product building, that is also a chance for us because we can interact more with our customers because we have to, there's no more choice, right? Before you may be able to hide between your behind your features, now there is no debate. If you if you don't go to them, if you don't speak to them, if you don't try to understand everything, you are not going to do your job really well.
Randy Silver 30:42
Let's get into that. But before we do, and just want to follow up on one thing. And with teams having to continue to monitor and maintain the things that they've released, uh, are you does it change the way that you, as a manager, as someone who looks at the organization, is budgeting for resource allocation? Do you have to say uh the amount of time you know the the team may release something, now they're free to move on to something else? Or are you really thinking, okay, how much more can they take on because they're maintaining two or three experiences and they have to make sure that that they stay strong?
Francois Lopitaux 31:20
Yeah, I mean, the the way that we have designed our organization is not really per features, and I think this is super important, right? Because you know, you have different ways to organize your product team. Uh, you can do by stack, you can do by features, or you can do by uh outcome. Or what I'd like to say is in our side, what we have designed is we have designed uh what we call by track, and each track is responsible for uh some a product. And so their responsibility is a product. Their responsibility is the every feature that they should build for this product to make it successful, which means like when they are building the features, it's it's a feature, not like you know, it's a feature about this specific product. So when they speak with the customer, for example, they can speak about the features that they have built like last uh last month, and they can speak also about the upcoming one that they are going to build. So it's not so much that it's going to change the workload or anything like that, because the way that we have designed the organization, it makes sense. They don't move from you know one thing to another things, it's a continuum, it's just like the continuity of the product that they're working on. And so it's not really adding new stuff or like, oh shit, I need to look at my you know, my my other features that I built. It have nothing to do with my new one that I'm building. No, it's all like in the same directions, so it's become completely natural as a follow-up. You will say, like, hey, you know, we deliver this new AI context features and they're generating context on top of your columns. Now we are doing this, we continue to improve the system. By the way, what did you think about the previous one? So, you know, it's completely like natural conversation, which I think is is really important. And also what is really important is because you create these kind of product organizations, then they are the best to decide what should be the next things to build. Because they are in the weed, they know exactly their customers, they know exactly their product, they know the pain point, uh, and they know what is going to go next versus me, which you know is managing multiple product area. I should not be the I am not the not the first person to know exactly which feature to to create next for each track.
Randy Silver 33:27
What about retiring and sunsetting things? Is that still you know, it's so easy to build now? Is are you paying attention with the teams to pruning and making sure that you know the surface of the product is the optimal size?
Francois Lopitaux 33:42
Yes, I think that's I love to prune, by the way. Uh it's my I love, I'm a I'm a freak in terms of organization and I love to kill stuff, I love to remove previous features. So, you know, you have no choice. Like, for example, for our starter, uh, our conversational ethics capabilities, we created v1, we have created v2, we have created v3. And so if you don't prune your stuff, you again you are going to end up with uh some kind of like monster where every time you add something here is going to break something there. So pruning, I think you know, pruning was important in the previous world, but pruning is becoming even more important now because again, it's back to the organization and and you know the fact that you need to be super strict because the code generation is much more easy to do. You need to be careful that you know it's like a tree, right? You want to have a branch to go in every direction. You need to know when to have to cut a branch that your tree is growing to the right directions.
Randy Silver 34:41
Francois, you talked a couple of minutes ago about you know an engineer being able to do the product manager's job and vice versa. And I'm definitely seeing that, but I my take on it is that you know, I can do 80% of an engineer's job, and I can do 80% of a designer's job, and a CEO can do 80% of all of our jobs. And the fundamental difference is, you know, if you give me the sitting, me and a designer and a developer the same tools, we're gonna use them differently. We're gonna do different things with them. I think it's more of an attitude than anything else. And I'm just curious, you know, we used to say a product, the debate a couple of years ago was how technical does a product manager need to be? Do the do you need to know how to code? What is the the right question now? What is the core skill set that you're looking for? And are you hiring product manager people? Are you hiring attitude people? Are you hiring people from different what what do you want from your team?
Francois Lopitaux 35:38
So I think that's my requirement. One of them never change. I strongly believe that you need to hire a technical product manager. Because if you don't know the basic, so I'm not saying like you should be able to code or whatever, but if you don't know the technology, if you don't know why it's working, it's going to be super hard for you to help your engineering to deliver features and to understand like the nuance and to understand you know why something is complex or not complex, what is possible, what is not possible. So I think like technicality for a product manager for me has been require number one since day one, since you know 20 years ago. Never change. Now, what changes uh and did it change? I I will say I'm not sure yet actually, but the second things that I'm really looking for is curiosity. Like the my PM, the one I'm hiring, I need people who are curious, who want to understand, who want to learn the latest technology, who all the time thinking about how they can improve some things, how they can feel like you know what customer needs, and so on. And I think if you if you want to be able to really understand like customer pain, what they want, you need to have a lot of empathy, you need to have a lot of curiosity to do a good job. So I think that the really the two main aspects I'm looking for uh for product manager when I'm hiring. And very recently, actually, I I hired a new associate product manager, and what I put on LinkedIn, uh it was very simple. I I say, like, please, you know, send me your Git repo, because now everybody can code, so every PM should have a Git repo. Secondly, send me a video that is explaining to me why uh your latest things that you have coded on your Git repo uh make sense, solve a problem, and is kind of like you know, your product, right? And I think like doing this with this approach, you are really focusing on the fact that the person is technical. You are focusing on the fact that the person is obviously great at public speaking because he's going to preach present his project and he's going to pitch it, and also great to understand that pain point and to solve it. So I think that's really what I'm looking at uh for PM, but technically, you know, big technical has been number one for me for a very long time.
Randy Silver 37:56
And we talked for a long time about uh the optimal size of a team being, you know, a two-pizza team. And now we can do I'm seeing two different trends. I'm seeing teams shrinking in size a lot, or uh sometimes at larger companies, I'm seeing teams stay the same size, but the the the attack surface, you know, the amount that they're expected to cover is is much wider. What do you think a good team looks like? What are you trying to uh build these days in terms of an organizational unit of a team?
Francois Lopitaux 38:26
I think the team is going to shrink because you want, you know, the bottleneck, coding is going faster, designing uh a little bit also, but the biggest issue that we need to solve now is what is meaningful for customers, what feature is going to move the needle, what is really going to make us win the market? And this uh you still need you know time to think about it, and this is really like the job of the PM. And so because execution is going faster, I really think that the team are going to decrease, and the ratio PM developers is going to decrease. And the two pizza team is not going to be the reality anymore. And also, even like some developers may become like you know, a type of PMs. And some developers definitely, I see in my team, like are very uh connected and kind of like do the uh you know part of the job of the PM. But I think these days now it's becoming even more important to know what you want to build because then after that, you know, you have to maintain it, you have to support it, you have to kill it if it's not working. So that's why you know spending time to be sure what you are doing is meaningful is becoming critical.
Randy Silver 39:40
Yeah, I've seen some people argue for the one-person team or you know, the happy mail team. And I don't think that's quite right either. But I haven't come up with the right uh food metaphor for the size of a team that's that's small but not one person. Francois, this has been fantastic. Thank you so much for your time. I think we've got time for one last question. So I'm gonna ask you to put on your wizard hat and and help predict the future a little bit. Um, we're seeing things change a lot in terms of cost. You know, the cost of LLMs and AI-based development has been hugely subsidized. It was very cheap for a while, but we're also spending a lot on tokens. Different models are coming out with different approaches. We're seeing uh frontier models at a certain tier, we're seeing open weight models come, we're seeing people build their own things and uh start their own data centers again. Where do you think this is going? Where do we what's the trend that you think is going to happen for the next few years for teams and companies that want to minimize their spend but still get real value?
Francois Lopitaux 40:41
I think that uh flexibility, I think, is a is a word. You want to be able to switch model super easily because you know today is you know, today is like um fabble, tomorrow is going to be like uh an open source model. We saw with Jeff, JEV now can solve problems of classification at a fraction of the cost of any other model. So, you know, basically you want flexibility, you want to be able to adopt any type of technology that is going to make sense for your use case as fast as possible. So I think that more than ever, you know, it has been all the time the debate about uh mega vendors versus niche product. And all the time the bad side of the mega vendor was like you are stuck with one mega vendor. The good side was everything is integrated and blah blah blah. I think for the LM and all this technology, you want to really uh be able to be flexible, pick wherever makes sense at the time, be able to switch super fast, because you know, yours of 2025 was to prove AI is working, to prove that the value was there. 2026 is about going to be now you get the value, you want to get a good cost in front of it, right? You want to have an ROI that makes sense. You don't want to spend more on something that don't bring you so much values. So I think that 2026 and 2027 is really going to be about optimization of this cost. And now with all the open source model, all the new techniques that you know, with challenge that we saw like this uh last Friday, it's it's really like uh possible. And it's I mean, you know, the opportunity is there. So you need to be able to catch the opportunity to use all the time the best option for you.
Randy Silver 42:37
Yeah, the more things change, the more they stay the same. We went through this with cloud optimization, we went through this with any number of other things. Um, yeah, it's just that time in the maturity cycle, I guess.
Francois Lopitaux 42:48
Yeah.
Randy Silver 42:49
Fantastic. Francois, thank you so much. I've learned a ton from this conversation, and I really appreciate your time. Thank you very much.
Lily Smith 42:58
The product experience hosts are me, Lily Smith, host by night, and chief product officer by day. And me, Randy Silver, also host by night.
Randy Silver 43:08
And I spend my days working with product and leadership teams, helping their teams to do amazing work.
Lily Smith 43:14
Lou Ron Pratt is our producer, and Luke Smith is our editor.
