Dual-Track Agile: Why Messy Leads to Innovation

BY JACOB DE LICHTENBERG ON APRIL 4, 2017

What is Dual-Track Agile?

Dual-Track Agile is an IT development  methodology where figuring out what to build is as important as the building process. You start with a discovery track to find out if a product idea is good and if it makes sense to build. Successful findings from the discovery track are added to the backlog of the delivery track.

So how do you know if your product team has a good idea? Luckily, there are a variety of methods that you can use to figure it out. Personally, I am a fan of the experimental approach, because you evaluate ideas based on behavior before you invest in them.

With Dual-Track Agile, the development team participates in the discovery track – often led by the lead engineer, the UX designer, and the product manager. In other IT development methodologies, the IT teams are only a part of the delivery track, so they don’t get the valuable knowledge from talking with users and performing experiments.

The Dual-Track Agile process is demonstrated in the gif below.

Dual_track_agile_Jacob_de_Lichtenberg.gif

Now Things get Messy

While Dual-Track Agile is my recommended way to develop products, it’s not very clean. Think about classic project management, and how it’s anything but easy. Now step it up a notch with an extra work track for performing research. Dual-Track Agile raises complexity in many areas, especially when it comes to planning and resource allocation. Since most product managers are already familiar with delivering production-ready code in the delivery track, I’ll keep my focus on the discovery track.

How to run a Discovery Track

The purpose of the discovery track is to work out if a product idea makes sense to build. During the discovery process, the product team should collect answers to the following four questions:

  • Will users use it? (Value)
  • Can users use it? (Usability)
  • Can we build it? (Effort)
  • Can we get internal support? (Stakeholders)

Let’s take an example from product development at Trustpilot, where I work as a Product Manager. Trustpilot is a review platform that helps companies collect reviews, and consumers shop smarter. Reviews are displayed publicly, so consumers can read about companies before they buy products or services from them.

Objective and Brainstorming

At Trustpilot, we get an overall objective for each quarter. The objective is outcome based (e.g. increase leads) rather than delivery based (e.g. build a specific feature). Outcome-based objectives are fundamental to Dual-Track Agile product development.

Our objective in this example was to increase retention.

The team, comprised of developers, designers, and myself,  came up with a variety of ideas that could help us reach our objective. One of the ideas we came up with was the digest email.

The idea was to send the digest email to users two weeks after they had written a review on Trustpilot, to provide them with an overview of how many people have read and liked their review. Digest emails work well for other companies, but we had to work out if a digest email would make sense for Trustpilot. Would it give users significant value? We used the process of discovery to reach an answer.

Framing an Idea

Before discovery, we had to frame our new ideas to ensure that we understood the basics of our product concept. In the framing process, we  rejected some ideas because we could not argue that they were important enough, but eventually, we came up with this frame:

FRAME “Digest email” idea
Why are you building this? To get users back to Trustpilot’s website after writing a review (retention).
Who is it for?

[What is the minimal customer group]

All users, but especially frequent users.
How would they use it? Users will receive an email, so they know how many they helped by writing their review. This praise will motivate them to return and even use the platform more.

There are more detailed ways to frame your ideas, like the opportunity canvas, but we chose to go with the three questions above, as they are the core in all idea frameworks.

Once we finished setting an objective, brainstorming, and framing the idea, we were ready to move on to discovery.

1) Value: Will Users use it?

The first question is if users will use or buy your idea. It is rarely a question of whether an idea creates value,the question is how much value the idea creates. In some cases, the question could be how much customer are willing to pay for what you are developing. At Trustpilot, our question was how many users would return to the Trustpilot review site after receiving our digest email? And the easiest way to test this out without building anything? We faked it.

To fake it, we randomly selected 5,000 Trustpilot users.

We created a mockup of a digest email, shown above.

The challenge with  the mockup was that we didn’t have accurate statistics on who had read and liked reviews of specific users.

We could have made up numbers, but that wouldn’t have been ethical. (Ethics often get messy with Dual-Track Agile, but that’s another topic.)

However, we did have the tracking statistic for how many people loaded a page on which the review might be shown. So we were able to estimate numbers. To have a clear conscience, we even put our uncertainty in the footer of the experiment email:

At Trustpilot, our principle when it comes to running “will they use it” discovery is that we validate our ideas in the fastest, cheapest way possible. So to keep costs down and to move efficiently, we removed the more complicated part of the email which displayed reviews from other users with similar opinions. That resulted in this simplified version:

Results:

After sending out 3,000 emails with estimated statistics, we got the following results:

Email opens Retention Re-activated
52% 21% 5%

We found these results very promising, and our verdict was that our idea also provided users with significant value.

2) Usability: Can Users use it?

It is extremely important to talk to the end users who will be using your product.

The usability question in discovery is typically whether a user can understand and use a complex product or feature. The trick is getting the design right. This can give great insights into whether a product or feature will work. In Trustpilot’s case, the idea was simple, so we wanted to know if users understood the email and how it made them feel. In user interviews, we presented the following mockup:

Being presented with this, it seemed as though all users understood the email, and we had users saying things like:

  • “I hate to admit it, but this kinda makes me feel important.”
  • “It makes me feel popular.”

One advantage of interviews is that you learn more than you ask for. And we definitely did. Turns out, the placement of the “found useful” number below the number of people who read the review was quite problematic. Users were disappointed that of the 36 people who had read their review, only five had found it useful. The “found useful” took value away from the “reads”.

Results:

All users could use the digest email, and they thought it was easy to understand. The email content made them feel important and popular, which confirmed the value of the idea. We decided to remove the “found useful” numbers from the email to avoid deflating the value.

3) Effort: Can we Build it?

This is more of an engineering task than a product management task. Though it is seldom a question of whether product teams can build something, but rather how much effort it takes to build both in relation to man hours and finances.

We knew that our digest email was not the simplest of tasks and it would take a couple of weeks to build. But based on our prior findings, we concluded that it was a feasible investment.

4) Stakeholders: Can we get Internal Support?

I don’t like the words “stakeholder management” because it sounds like you have to persuade people when really it’s about getting input from others who have more information than you. For example, legal, sales, marketing, and support departments, all have valuable knowledge that you need to take into account. These stakeholders may present information that you didn’t know about, and in the worst case, this information might have to stop the experiment altogether.

In our case, we presented the digest email idea to our sales and legal departments, as well as top management. Honestly, we had expected everyone to just say ‘yes’.  But it turned out that there were two concerns, namely a risk with our enterprise customers who might be reluctant to share data. Secondly, we couldn’t send the email to users in all countries because of different email/marketing laws.

Results:

We decided to push forward with our idea, but we added an opt-in to the digest email being sent to users in four countries with strict marketing laws. And to avoid scaring enterprise customers, we decided that we’d roll out the product gently so we could fix issues fast, if any would emerge.

Moving to Delivery

After having positive results from all four questions in the discovery track, we decided to give the digest email a ‘go’, and moved our idea from the discovery track into the delivery track.

There were several weeks of building, followed by usability testing, and in the end, we had a real digest email geared for Trustpilot users.

We saw positive results immediately. After receiving the digest email, more users started going back to Trustpilot’s website each week:

We even got great feedback from satisfied users on social media:

 

Why is Discovery so Messy?

In theory, adding a discovery track to an agile process is clean and useful. But the strength of Dual-Track, namely research and avoiding expensive failures, can also be the weakness of the method. You see, pure delivery is like a factory because you simply have to estimate how long it takes to produce code. But, when you add a discovery track, you make staffing and planning more difficult and complex. That might make some people in your organisation raise their eyebrows.

Making a Discovery Plan

You might be able to ease the minds of managers or other colleagues by showing them plans for your feature or product ideas. But if they expect to see traditional road maps or timelines, then you’ll have to disappoint them. With discovery, you won’t know what experiments will be a success and your focus has to be on the outcome. For an organisation used to measuring written code and value points, this may create an uneasy feeling of unpredictability.

And discovery can create even more entropy. Sometimes learnings from a discovery experiment can require new experiments, in which case we need to extend the current discovery process and run more tests and interviews in order to verify the product idea.

This messiness is an inherent part of Dual-Track Agile – because you plan with objectives as opposed to features. Sometimes upper management may insist on a roadmap because that’s what they’re used to. In that case, I have a few suggestions.

One option is to map out the experiments that you’d like to perform in order to achieve your objective. That doesn’t mean creating a timeline, as that is quite tricky to estimate. It depends on traffic/volume on the test page, how close results are before reaching statistical significance, and the complexity of the test. Depending on which part of the product you’re experimenting with, it can take anywhere from an hour to several months.

Another option is to create a long term plan of objectives/themes in collaboration with upper management, so they have some input and control of the process.

Organising Staff 

Staffing is perhaps the messiest part of Dual-Track Agile. Having multiple tracks creates complexity because team members have to switch between different tasks faster. My team at Trustpilot appreciates when we can work on the same task together, as it makes them feel interdependent, it’s motivating, and it’s efficient.

The moment you start working on a discovery idea, you split the team up a bit. And since discovery tasks are often small, teams are usually divided into smaller groups or tasks. This can be challenging to manage.

The discovery process can become more chaotic if many ideas fail. Failing is of course the strength of Dual-Track Agile because failed experiments are much cheaper than releasing features with no value. This ‘fail fast’ process is also educational, as you learn how to build better products.

But sometimes you might run out of delivery tasks in the backlog, and the whole team can work on discovery together. This could be fun if everyone gets to collaborate on the same form of discovery. Then again, keep in mind how you use the precious time of developers.

In Trustpilot’s case, most of our discovery is made in the front-end because we experiment a lot with user experience. That means we don’t really have discovery jobs for our back-end developers. Instead, they take advantage of the time, by e.g. repaying of tech debt. It creates value, but it divides up the team too. Overall, Dual-Track Agile makes staffing challenging and requires pro-active developers.

Why I Always go With Messy

Despite the increased complexity and chaos, I’d still choose Dual-Track Agile any day over other traditional methods. Here’s why:

  • You only build a product that you know is valuable for real objectives, and therefore have significant business impact.
  • You never start to build something that does not turn out to be valuable. This includes HiPPO ideas, which often get built but then fail with no one to blame.
  • Discussions last forever if we have no data. We can kill ideas pretty fast. A discovery experiment kills or thrills an idea instead of having endless meetings.
  • The development team is involved in business decisions. This way, they understand why you’re building a new product, so there are fewer misunderstandings and in turn, fewer mistakes. It’s motivating for development teams to have an influence on product decisions. Motivated staff are more efficient.
  • You learn more than you expect. So instead of just being a factory, the team gets better at building products, and engineers are really clever people who then end up coming up with the best ideas.
Jacob de Lichtenberg

About JACOB DE LICHTENBERG

Jacob de Lichtenberg is admittedly obsessed with building successful products, and therefore hooked on product experiments. From Citrix, CSC, to Trustpilot, Jacob's product management focus has always been on making innovation fun and thriving for his teams. You can read about some of Jacob’s thoughts about experiments on on https://medium.com/the-experimental-approach.

RELATED CONTENT

Read more about Agile ...

Planning Your Prototype With a Customer Journey Map

  • I appreciate you taking the time to write up how this works at your company. I’d love to see more product teams sharing how they work.

    Be careful about thinking about discovery and delivery as separate phases. This leads you down a path of mini-watefall where you still have problematic hand-offs and team members not having the necessary context to make good decisions. It’s also a much slower way to get to a viable solution.

    Instead of thinking of dual track as two phases where one follows the other, a better visual analogy is to think of it as a braid with two strands where discovery and delivery are intertwined. Discovery feeds delivery. Delivery feeds discovery. Continuously.

    • I’m really interested in why ‘braids’ and not tracks? Discovery will always feed delivery although in our case, delivery teams are involved in discovery which breaks down the issue you mentioned with hand-off (Jacob said they do the same). The only feed back into discovery from delivery is feedback from the released code, is that the feed you are talking about? I drew out structure here if it help clarify my point I’m really interested in why ‘braids’ and not tracks? Discovery will always feed delivery although in our case, delivery teams are involved in discovery which breaks down the issue you mentioned with hand-off (Jacob said they do the same). The only feed back into discovery from delivery is feedback from the released code, is that the feed you are talking about? I drew out structure here if it help clarify my point http://playbook.co-innovationlab.com/images/dual-track-agile-diagram.png

      • The value I see in the braid analogy over the tracks analogy is that it encourages tighter back and forth cycles between the two rather than a one way feed from discovery to delivery. It sounds like everyone agrees that discovery feeds delivery. If your product is well-instrumented, then delivery should also feed discovery. As you ship code and measure impact, you’ll find gaps in what you expected and what you measured. These gaps become your inputs back into discovery.

        If we think of discovery as a separate track, we can fall into the trap of long discovery cycles, iterating too many times toward a perfect solution, where in practice we might learn more by simply shipping something and measuring the outcome. The braid represents a rapid back and forth between discovery and delivery. Learn a little, build a little, learn a little, build a little. It’s the same idea as rapid cycles through the Build-Measure-Learn cycle from Lean.

        By the way, my original comment was in response to the way Jacob illustrated the process rather than how he described how his team worked. In his visuals, discovery happens first followed by delivery. This depiction of dual-track as a linear process where one follows the other is misleading. And Jacob, I agree, it is precisely this back and forth iteration that makes discovery messy and why I like the braid analogy better. 🙂

        • That’s really useful – one of the issues I have been plagued by in the past is exactly that, long discovery cycles; depicting and thinking about frequent releases and tight feedback loops in a ‘braid’ helps a lot so thank you 🙂

    • Jacob de Lichtenberg

      Hi Teresa. Thanks for joining in. I am sorry if the discovery work and delivery work do not come across as intertwined. In the way we work, they are and we avoid a classical handover. This is what creates part of the messiness. I would actually think that if they aren’t, we would most likely have a ‘partial-team’ discovery process (anti-pattern 3: http://svpg.com/product-discovery-anti-patterns/). We actually have a rule that a developer has to participate in all discovery work. This helps both in getting ideas and for developers to get the small details we learn in e.g. interviews.

    • Gino

      What do you mean by mini waterfall?

      For me waterfall is
      – long delivery times.
      – handovers.
      – no feedback loop with the customer.

      I try to prevent mini waterfall by
      – bringing engineering and UX into the discovery process.
      – I try to keep the cycle time of a problem to solve by making it smaller.
      – I involve users and customers through the whole process.

      • Waterfall is also a step-by-step sequence where one team does one piece and then another team does another piece. Marketing identifies needs, product writes requirements, designers design it, engineers build it.

        If we think about discovery and delivery as linear processes, we can easily fall into the same trap. Where some people discover and others deliver. Even if it’s the same team, we can still run into the same slowdowns where delivery waits for discovery to be done.

        Based on the follow on comments, I suspect that Jacob’s team doesn’t work in a linear process at all. But I know that many teams do, which is why I prefer the braid analogy, as I think it better illustrates the back and forth movement between the two.

        • Gino

          So having the full team working on discovery and delivery together, which can cycle back and forth, is a better approach? If so, I agree.

          Splitting it into two teams creates a dependency on each other. You want each team to be able to deliver efficiently. So we have UX and Product in our squads and started to add our discovery and launch tasks into the teams backlog which can be picked up or supported by anyone.

  • LaRoy Page

    I use much of this same thinking with a tool I call the Critical Effective Path. The CEP provides a roadmap where each vertical is geared toward answering a difficult product or technology question or validating a primary assumption. We may use the Google Venture Sprint concept or other discovery methods to get the answers and feedback we need and will usually follow with a delivery to set us up for the next vertical. Using this method we can often deliver value well ahead of schedule and usually end up needing to deliver much less than was envisioned to attain the desired overall impacts.

146 Shares
Share
Share
+1
Email
Tweet