Product Diligence: The New Technical Diligence – Part one

I was surprised to hear recently from a startup friend that no one did any technical diligence aside from interviewing their tech lead before securing their multi-million dollar round. But let’s face it, technical due diligence is a lot harder than it used to be with Agile—there aren’t giant PRDs and Gantt charts lying around, and code evolves so quickly that pretty much every app just became obsolete right….now.

I propose a pivot from technical diligence to product diligence. The difference is instead of merely diving into code and interviewing developers, we change our view to focus on the product strategy, development, and management practices.

Specifically, we look at the following:

1. Is the product unique, ie. protectable intellectual property (IP)?
2. Does the company have a solid product strategy?
3. Does the company have a rapid product development environment and culture?
4. Does the company have technical debt that will cause a reckoning?
5. Can they scale their product quickly?
6. Is there anything else about their product that proposes a risk?

Of course the importance of each of these will vary depending on the nature of the company you are looking at. Some B2B apps may not need to handle massive amounts of traffic. Some business models don’t require any IP to have intrinsic value, etc. So the first thing is to determine how relevant each of these questions are and prioritize. Then you need to spiral into each question to make sure you need the answer. This will entail someone who has high-level technical chops, a CTO type if you will, as well as one or more people who can drill into the more technical aspects at the code level.

Whoever does this eventual analysis, each of these elements should be evaluated. Let’s take a look at how to approach each of them, starting with the first three.

Intellectual property

One of the more challenging questions that should always be posed to startups is: what makes you the right team to do this? And while technology is not necessarily at the heart of the answer, it might be. Thus, the key is identifying what the secret sauce of a company is, and whether they have written that code (or created the hardware) to support it. There is absolutely no problem knitting open source tools together to create something new and different, and in fact, many successful startups have done just that.

When investigating this, it is important to note that the company is aware of what their core differentiation is, and that they are focusing their development energies on that. (Wardley mapping can help with this).

To give an example, I did some work with a software-as-a-service (SaaS) app. Their core focus was mainly around managing their customers. However, they spent considerable development time building things that are commoditized: homegrown analytics, homegrown database migrations, etc. So while they had considerable IP, it wasn’t necessarily valuable. Companies should be focusing on building what is their core value, and that is where their IP should be.

How To: IP Diligence

The best process to discover this is as follows:

• Review the issue management software and release notes to find out what is being worked on, what is in the backlog, and what has been completed. This should give you a feel for where they are focusing development energies.
• Interview architects and development leads to discover what they are writing, what frameworks and libraries they are using, and what technology. The focus here should be on understanding what code they write and what they’ve built.
• Review the codebase. I realize not everyone is qualified to do this in all environments, but it’s important to review the code to see how well it is written, if it is understandable, well-formed, and to validate they actually have something of value that is being created.
• Gotchas: Building things from scratch that should be easy to implement. Nobody can articulate what their core product IP Is. No central process or person for architecture.

Product strategy

What we are solving here is whether there is a comprehensive strategy to what is being built. Are things being prioritized appropriately? Do they have a general “roadmap” for the future? Are they validating what they build with users, in other words, is the backlog evidence-based? Do priorities change frequently? Who is ultimately running the product strategy: customers, sales, marketing, engineering or a product team? All of these will give you a clue as to whether they are doing it right.

Ultimately, what you want to see is a company that values product management, a gatekeeper who resides between business needs, engineering, and the customer. This should be in balance. Doing everything the customers want can lead to a messy product and not proactively shipping features that exceed their expectations. Doing everything to drive the business desires without validating with users can result in missing product-market fit. And engineers should never be driving features and UX—though their input can be invaluable.

I recall one company that I worked with whose product managers were often on site with the clients. Since they were pivoting from smaller clients to an enterprise play, they were very interested in what these new clients had to say, and for good reason. However, they would not only promise features to the client on site, but also give them dates when they could expect these features (without consulting engineering, of course). The result: spaghetti code, low developer morale as they were driving fast to unnecessary deadlines, and the surprise kicker: unhappy clients. The latter was true because they never had time to be strategic, to collate what the clients were asking for into what they really needed. So they ended up just building what they said, but the result was often late and buggy because of their process.

How To: Product Strategy Diligence

The key to this will be to get what you can from the product team:

• Check the org chart: they must have a product team and in the right place on the org chart. At least one product manager or head of product who is providing strategy and not just tactical backlog grooming and team management. Also, they should not be in charge of engineering or marketing but adjacent to them. (Sidenote: A CEO often has input —especially a founding one—but they should not be running the product.)
• They should have some kind of roadmap. I am personally not a huge fan of Gantt chart style roadmaps because they give an illusion of time management, and no highlight of the possibility of change. At least a now/next/later list of features and/or quarterly or monthly roll out plan.
• Talk to the product team to ensure that features are being validated. There should be a clear process of validating this information with users through analytics, user research, and/or A/B or multivariate testing. Building minimal features and apps to test in a Minimum Viable Product fashion is a great sign.
• Finally, discover what the overall product management practice is about. Strategy should be a part of it, not just tactical.
• Gotchas: No product team. Product team is on sales calls. No roadmap. Overly structured roadmap, or hard deadlines with fixed scope. Complaints about “QA issues” are invariably problems upstream—usually in product. Complaints from marketing, business, engineers or designers about the process – all of these should be considered and included in the product strategy.

Rapid product development

Ultimately, a startup needs to be able to ship products fast. One of the best books written about the software development process is Accelerate. It’s great because it involves actual research on what makes product teams successful. Ultimately: speed. Speed is achieved by having a good development methodology (ie. Agile, Scrum, Kanban), using a CI/CD pipeline (aka DevOps), writing loosely coupled code, and taking advantage of frameworks and open source libraries. It is pretty easy to spot a well-oiled team, however it is also a rare beast. Keep in mind you will get deeper into the scalability of the product later, but for now you are just evaluating whether they have the process and architecture in place to support speed—and whether it is working.

Watch this #mtpcon Digital APAC keynote by Cameron Adams, Co-Founder and Chief Product Officer at Canva about the 3 stages of product development at a high growth tech company.

How To: Rapid product development diligence

• Evaluate their issue tracking software. Are user stories linked up to epics and is it easy to track what is being worked on and estimate when it might be finished—at least what percentage is complete. The inability to track progress is usually a bad sign.
• Evaluate their CI/CD pipeline. Do they have one, and what is included: anyone can spin up a dev server, auto-deploy to developing/staging/production, code linting, running automated tests, etc. This should also include a comprehensive git process of doing pull requests and code reviews, merging, etc.
• Are they running spikes to evaluate potential solutions or to find the optimal library, framework or 3rd party tool to perform future feature implementations. This is a sign that they are thinking ahead, focusing on building what’s right and using the right tools.
• Are devs empowered to grab any ticket at the top of the backlog when they are ready, or do they have a lead dev who must always be involved, or does each dev focus in a particular area? While sometimes it makes sense to consult a lead dev, the fastest teams are self-driving and can work anywhere in the codebase and no single engineer is always in the critical path.
• Is the code loosely coupled? For starters, there should be a clear backend and front end using established and active frameworks. Ideally, each should also then be written in microservices and architected such that everything doesn’t depend on everything else to work. You will need to interview engineers individually to get a candid opinion on this, as well as review the code or have it reviewed by an expert in those frameworks.
• Gotchas: No CI/CD pipeline and/or manual release processes. High team turnover or low morale is a good sign something is broken here. If devs and others agree that things take too long, they probably do. Using an old framework may be a red flag, as does a mono-repo (single git repo for all code). Devs spending the majority of their time fixing unit tests is a sign that there are too many dependencies. Code duplication (a number of tools can be used to find this) are signs that a lot of copy-paste is happening.

Conclusion

As you can see, by coming in from the angle of product instead of technology, we are taking a more holistic look at how product is conceived as well as developed. In this first part, we discussed analyzing intellectual property, product strategy, and rapid product development. In the next part, we will discuss technical debt, scalability and managing other risks.

Discover more content on the Product development process. Engage with further insights using our Content A-Z.