In the world of venture capital and private equity, due diligence is a critical aspect of mitigating risk for investors. Historically, when investors do their diligence, they do technical diligence along with financial and other forms. While I think this is important, my experience consulting with many companies of different sizes has shown me that often, the root of technology problems lies not in the technology, but in poor product management practices.
Enter product diligence. Through this concept, we analyze not just the product itself but the processes and methodologies behind why and how their product gets built. In the first part of this series, we discussed intellectual property, product strategy, and rapid product development. In this part, we are going to dive more into the dangers: technical debt, scaling and other risks.
Technical debt
The biggest enemy to shipping features is technical debt. It’s also a reality in every piece of software ever written. It ranges from needing a patch to needing a ground up rewrite. Every product manager is constantly negotiating how to manage these issues in the optimal way to not disrupt new feature rollout—or at least they should be doing that. To become aware of this problem, find out if they are tackling this in their issue tracking system and talk to the devs. In my experience, devs hate technical debt: every piece creates potential for breaking the current system and represents more work for later.
One startup I worked with was a banking SaaS that had accrued so much technical debt that our team had to do major work just to get them back in compliance. Parts of the site were written in React, others in Angular, others in vanilla JS. Every PHP script we opened up revealed new problems, commented out code, hacks, and kludges. We managed to get them back in compliance and improve the code, but everything took double the time (at least) to get this done. This is a hidden risk that can bring down an entire company because they are no longer able to ship features as fast as others in the marketplace.
Read this blog post on A Product Manager’s Approach to Building Integrations for SaaS Software.
How To: Technical debt diligence
- Talk to the product managers and lead engineers about their approach to technical debt. If they have no approach, then they have technical debt.
- Talk to the developers—all of them (anonymous surveys work). Do you have technical debt? How much more time does it take to work around existing code? Do you typically fix things while you are in there or is there a separate process, or none at all? Are there any frameworks that need to be upgraded?
- Inspect the code. A few key elements are giveaways: commented out code, comments saying things need upgrading, different frameworks in the same code, monorepos, and more.
- Gotchas: behind on major upgrades like the latest PHP version that will cause deprecations. High team turnover and low morale. Complaints of things “moving slowly”. Complaints of long testing cycles as new code is wrapped in many unit tests to hide dependencies. Lead devs and architects who are not up on the latest frameworks are a good sign that things might be backwards or outdated.
Scalable product
If a company is going to grow fast, the technology has to accommodate it. I define scalable both as being able to scale as new customers are onboarded, but also being able to scale as new devs are. So really this has a couple of different angles: performance and also developer-friendliness.
Watch this AMA on Scaling Empowered Teams: Advice From Nilan Peiris.
One company I worked with had an application that failed on both counts. This product was written using an old application server framework and vanilla javascript. Post Series-A, a new team of developers came on, and it took them weeks to do any kind of substantial work in the code. Additionally, things that come out of the box in newer frameworks (like database migrations) had to be hand-coded, further complicating the codebase and making it difficult to create a CI/CD pipeline and development environments. To make matters worse, some pages took up to 30 seconds to load, while others that were able to load with smaller datasets simply timed out. There was no concept of pagination or lazy loading. The site was a ticking time bomb.
How To: Scalability diligence
- Analyze the platform: is it modern and up-to-date?
- Interview developers to discover how long onboarding took them, and how long they think it would take new developers.
- Do a pointing exercise with developers: write up a simple feature description and have developers estimate how long it would take to create it from scratch and how long it would take in the existing codebase. Ideally, it should take less time to do in the existing codebase—at the worst-case, it should be the same. If it takes longer, that’s definitely a sign of trouble.
- There are many different tried and true methods for performance, load, stress and scalability testing, from Apache JMeter to newer SaaS based tools. Your objective isn’t to find out what the breaking point is, but to find out whether it is good enough for the stage of the company and projected user levels. For example, if a company is at Series A and is predicting 1M more DAU in the next year—the app had better be able to scale to that level in a reasonable fashion.
- Examine the cloud settings for the app—is it written using serverless technology that can scale infinitely? Does it use auto-scaling, or will they have to throw more instances at it as the site grows, and how long does that take? Do they use a CDN? Again—is this reasonable given the business model.
- Gotchas: high developer turnover or complaints from anyone about development taking “too long”, issues that consistently take more than a sprint (2 weeks) even if they are given five or fewer points, slow loading pages, missing normal performance practices like lazy loading or page loading before content has loaded and pagination, a lot of tickets about slow web site or unavailable website.
Other risks
There are several areas where things can go wrong with a product. When you are evaluating, be on the lookout for these other possible red flags:
No or small QA team. Ideally there should be at least one QA per two to three devs. Also should be automating if you are past the MVP stage.
Messy issue tracking systems. Trello, Jira, Asana – whatever – should be clean and follow a simple backlog >> in progress >> (code review) >> testing >> ready process.
Missing key ceremonies—standups, reviews, retros, planning/grooming sessions all provide key touchpoints for handing off with devs. If this isn’t happening, it’s a sign there is not a good connection between developers and product.
Missing key roles: product manager, scrum master or project manager. In early stage startups, this is normal—and often a co-founder is acting product manager, but Series A and beyond these roles need to be filled. Product managers should not be running tactical scrum ceremonies and is a sign there is a missing strategy. Anyone really can run agile ceremonies (ie acting scrum master)—but make sure they are doing it right.
Quiet collaborative channels. If there isn’t much noise on Slack/Teams, this can be a sign that there isn’t much collaboration and a lot of silo-ing.
High turnover or low morale in any area of product development.
The CEO won’t give up product post seed. Often CEOs have a product vision, but ultimately, they have a LOT of other stuff to do. They can still be involved with product strategy, but shouldn’t be doing tactical product management. In companies I have seen this happening, I have also seen CEOs come up short on funding rounds, miss key numbers and not having enough oversight on marketing and sales.
Cool Dad CEOs. Ultimately, CEOs are the limiting factor of a company, and it’s fascinating how companies will actually adopt their dysfunctions. But the most toxic CEO type for a product is what I call the “Cool Dad” archetype. This CEO avoids conflict and exerts little control over his senior management, “letting them run with it”. What happens is that these execs often compete internally, and it creates in-fighting in much the same way children might behave if their parents don’t provide boundaries and mitigate. Interestingly, I have not seen this as often with women CEOs which is why it is Cool Dad and not Cool Parent.
Conclusion
Overall, there are a lot of areas of risk in the area of product when you are performing diligence. But I think the old way of doing “technical” diligence really misses modern-day products which is a combination of design, marketing/growth, business, and technical. Contextualizing Product Diligence as a separate but essential aspect of due diligence will serve to identify risk in potential investments.
Discover more content on the Product development process. Engage with further insights using our Content A-Z.
Comments 0