Product testing post-launch is a double-edged sword. Here’s how to do it the right way
Product Growth

Product testing post-launch is a double-edged sword. Here’s how to do it the right way

July 29, 2026/8 min read

It’s been a long time since a product delivered that “wow” moment that captured the world’s imagination. Outside of software, where AI has been the notable exception, we haven’t truly had a moment like that since the launch of the iPhone in 2007, despite a lot of hype around a range of devices. Even the debut of humanoid robots for the home–something that would have been considered the preserve of science fiction until relatively recently–has been met with a mixed response. 

1X’s NEO Home Robot was launched at the tail end of last year as a personal assistant that will take care of everyday tasks, such as folding laundry and loading your dishwasher. At $20,000 (around £14,700) for early adopters, it is certainly not cheap. But the most interesting aspect of its unveiling was that you will have to train your NEO and let it learn from its environment. In some cases, that will entail the robot being operated remotely by someone who will be able to see and hear what is happening in your home, all so future versions of NEO can operate more autonomously. 

This is a far cry from the traditional way products have been developed, but it’s indicative of a growing shift. Broadly speaking, developing new hardware products has been a sequential process, often what’s referred to as a waterfall approach. The product is designed, tested in a controlled or laboratory environment, and then released to the market. Typically, there wouldn’t be any updates from that point and, if there were, it would mean a product recall or releasing a second version. 

Now, in an age of more products being connected, and increasingly incorporating AI, the world of hardware is beginning to resemble software with more of an agile, DevOps approach to design. It borrows more from the “move fast and break things” attitude, taking more chances and pushing the testing and feedback loop into the market following the launch of a minimum viable product. 

Why it’s happening

It’s an approach that is only going to become more prevalent, for a range of reasons. Chief among them is that products increase in complexity and the testing required to develop a finished article becomes prohibitively expensive and time consuming. It would be almost impossible to develop NEO to a place where it could deal with any conceivable eventuality through lab testing alone. 

At the same time, the ongoing costs for many products are changing. There are connections to maintain, software to update, and apps to run. That inevitably puts pressure on companies to get a product out there sooner in order to bring revenue on stream and begin recouping the investment made in the earlier stages. There are also cases where seasonality can make or break a product’s release, meaning deadlines have to be met, or competition–particularly from China–forces companies’ hands.

That is especially true in fields, like robotics, which are at the bleeding edge of technologies converging: you could be looking at a decade of testing before realising any value on your investments. So the impetus to launch quickly is clear, but there is also significant risk that releasing a product too early could massively backfire. 

Where it’s been done the right way…

There are plenty of examples where companies have been able to push testing beyond a launch or incorporate data collection for new features while the product is in the field. One of the largest has been in self-driving technology.

Tesla has obviously had a lot of bad press, but the way it collects data to support autonomous vehicles is a great example of where the process has worked well. 

The company collects data from millions of customer vehicles, which act as a fleet learning network. Drivers capture real-word scenarios through their cars’ cameras, sensors, and GPS and use Shadow Mode to record safety-critical events or specific edge cases, uploading anonymised clips through Wi-Fi for AI training.

Essentially, the car manufacturer put a proposition to customers: it needed enough data to take to regulators that would support the case for legislation for autonomous vehicles. It would have been impossible for Tesla to plan for every potential scenario using only a laboratory setting. So the company communicated what it was trying to do, made sure the necessary data collection and monitoring didn’t feel invasive, and ultimately got what it needed from the process.

… and the wrong way

There are, however, plenty of examples of where putting a product out too early has gone the other way. Perhaps the highest-profile example being the Humane AI Pin, a wearable device that could answer questions, take photos and videos, and send messages. 

The company behind the product had raised $250 million (£184 million) and the device commanded a hefty price tag of $700 (£515), plus a $24.99 (£18) monthly subscription. But when users got their hands on the product there were issues immediately. Tech reviewer Marques Brownlee summed it up: "This thing is bad at almost everything it does, basically all the time". It was eventually discontinued in February of 2025. 

It was a similar story with the Rabbit R1, which raised up to $36 million (£27 million) and left consumers underwhelmed. Many critics and reviewers described it as “half-baked”, citing its lack of features at launch and shortcomings with its LLM. Both the AI Pin and R1 were touted as the next iPhone–and maybe they could have been with a bit more time and development–but their ill-timed launches quashed their potential. 

What brands can learn about when to launch products

When a high-profile product fails on launch, it can often taint an entire segment. When Google Glass was released more than a decade ago, the whole concept of smart glasses was put back years, notwithstanding the much more successful revival they are currently having. 

The risk of that happening has seldom been higher. People are a lot more sceptical about technology now than they were 10 years ago and the barrier to impress is much higher. So, how can brands get it right? There are plenty of lessons to take from recent experience, but five particularly stand out:


1. A basic usefulness test: In the case of the AI Pin, one of the major problems was that it just did not deliver on any of its promises, nor, even, on its most basic functionality. Trying to be all things to all people ended up being detrimental. There needs to be an obvious utility or USP before a product goes anywhere near its intended customers, otherwise they are unlikely to cut it any slack. Drill down, understand core pieces of value customer needs from the product and focus on those.

2. Give customers a sense of ownership: Giving your customers a stake in the process can go a long way, particularly in a B2B environment. Involving them in Beta trials can build a level of excitement and offering a reduced price or more support provides the quid pro quo many will want in exchange for the time they need to take with the product. Done in the right way with expectations set up front, they feel like a stakeholder in the product’s development rather than a customer. 

3. Communicate along the journey: One of the most important aspects of Tesla’s self-driving process was that it communicated with customers throughout. The company explained what it was doing, why, and how it would benefit the people whose data was being collected. That helped bring them along on the ride to a clear destination and overcome the trust barrier that is an inevitable part of the company-customer relationship. 

4. Know what can be changed (and what can’t): In the gaming world, when Cyberpunk 2077 was launched, it looked amazing, but had so many bugs it was practically unplayable. However, after a few patches, it is now considered a masterpiece. Software can be updated, but hardware can’t, and it’s important to appreciate that difference. If the hardware is flawed to start with, it becomes much more difficult to make changes in the field. 

5. Provide incentives: It is easier for customers to forgive issues if they’re being asked to pay less or nothing at all–you can’t expect them to pay full price for an unfinished product. As part of our initial trials of nooku, an air-quality monitoring device, we offered to install the technology in 40 of a housing provider’s homes for free. Their team liked the idea so much, they then offered to pay for it to be installed in another 100 homes. If you go about things the right way and show good intentions, people are much more willing to help you with their time. 

It’s difficult to amaze people with tech these days–consumers are unlikely to be wowed by a flashy device that lacks basic functionality. And in a world where products are becoming more complex and even toasters have an internet connection, user involvement at earlier stages of development is only going to get more common. So brands need to think carefully about how they loop customers into a release.

Do it too early, and it may spoil years of time and resources invested in getting to that point. But there’s an intangible bit of magic that captures people’s imagination, and often it’s the human element that’s so difficult to include. Engage the market in the right way–even with a product that isn’t quite the finished article–and you could create a product that is more than the sum of its parts.