Rerun: Using retros to create continuous improvement – Matt Walton on The Product Experience "Product people - Product managers, product designers, UX designers, UX researchers, Business analysts, developers, makers & entrepreneurs June 06 2022 False Customer Feedback, Podcasts, Retrospective, The Product Experience, Mind the Product Mind the Product Ltd 7282 Rerun: Using retros to create continuous improvement – Matt Walton on The Product Experience Product Management 29.128

Rerun: Using retros to create continuous improvement – Matt Walton on The Product Experience

BY ON

Matt Walton was one of the founders of FutureLearn, where he scaled the product team and organisation from nothing to the significant player in online education that it is today. Along the way, he learned a lot about continuous improvement of both the product and the processes for building it – including the use of four different types of retrospectives. He joins us on the podcast to level up our retro game.


 

 

 

 

Featured links

Follow Matt on LinkedIn and Twitter | Matt’s website | Matt’s Mind The Product talk ‘Be in a Band, not an Orchestra: how to grow an Agile Product Team’ | Funretrospective’s ‘The Retrospective Prime Directive’ | Management tools at 15Five and Trello


Episode transcript

Randy Silver: 

Lily, it’s coming to the end of the quarter. You know what that means? It’s time for a retro.

Lily Smith: 

Oh, I love retros. They can be a bit tricky, though, get it wrong. And it’s a bit like Festivus from Seinfeld with the traditional airing of the grievances.

Randy Silver: 

I definitely been in a few of those. But we can do better than that. Right? You What kind of retro Should we run?

Lily Smith: 

You mean there are different kinds of fruit trees?

Randy Silver: 

Yeah. And I know that was a rhetorical question is, this week we’re revisiting our chat with Matt Walton about the four types of retros that he uses.

Lily Smith: 

Randy so right through my cutting ploy. But now that we’ve set this up, let’s hand the mic over to Matt. The product experience is brought to you by mind the product.

Randy Silver: 

Every week, we talk to the best product people from around the globe about how we can improve our practice, and build products that people love.

Lily Smith: 

Because it mind the product.com to catch up on past episodes and to discover an extensive library of great content and videos.

Randy Silver: 

Browse for free, will become a mind that product member to unlock premium articles, unseen videos, AMA’s roundtables, discount store conferences around the world training opportunities for

Lily Smith: 

mining products also offers free product tank meetups in more than 200 cities. And there’s probably Well

Randy Silver: 

Matt, thank you so much for joining us on the podcast tonight. We’re really excited to have you. First time I met you was actually the first time I curated a product tank. And you gave a great talk about orchestras and festivals. And I’m sure we’ll get to that later. But for now, and for anyone who wasn’t there or hasn’t watched the talk. Tell us a little bit about yourself. Who are you? And how did you get into product?

Matt Walton: 

Thanks, Randy. Really nice to be here tonight. I’ve been very much looking forward to spending some time in both of your companies around the sent me regularly. Have you got something to rant about. And I got back in touch and said I’ve got something to rant about. So me, Matt. So I’ve been doing I guess kind of product. for about 20 years or so I started my career at the BBC where they called everyone a producer. But I found myself basically working with software developers and designers to build new things at a time where it was kind of quite hands on and you were writing bits of HTML, yourself and so on. And so spent about 11 years at the BBC first on the public service side and then later working for the commercial arm. And whilst I was at the commercial arm, I got involved with this Meetup group called product tank. days when it was in a pub, and most of the conversation was about like, how do you explain to everyone in your organisation exactly what a product manager is, you know, as much of self help group, but I kind of felt, you know, I’d found my tribe. And so so that was when I, you know, first was termed a product manager and felt that, you know, finally someone had put a label on what it was that I did. And then after I left the BBC, I decided to kind of freelance for a bit and work for various different organisations. And whilst I was doing that, I stumbled into this amazing idea of a thing called FutureLearn. So for those of you that don’t know what FutureLearn was, we were born out of the MOOC movement. So MOOCs are massive, open online courses. And it’s when, back in 2012, there was a lot of American University started dabbling with putting courses online. And so the, the Open University, which is an amazing precious institution that was born in the 1960s, to basically take education to more people than I’ve ever had it before, through the mediums of TV and the Postal Service, decided that this movement was quite an interesting thing that maybe they should get in on the act on. So I got brought in, in the very kind of early days when a guy called Simon Nelson had been hired to be CEO of this startup, and he was someone that I’d worked with at the BBC. So I got brought in to think about what a vision for the product might be, and put together a plan for how I might deliver it. And before I knew it, I was hiring some software engineers building a prototype and committed to launching an MVP in September. We’ve I’ve now grown the product team. So the organisation is about 200 people, but the product is about 75, including technology design and product management. And we’ve now delivered courses to over 14 million learners around the world that we’ve signed up over seven and a half year period, and about 18 months so ago, we got 15 million pounds of investment from the seed group or a bit like an Australian LinkedIn. And so that basically valued the business at 100 million pounds. So we’ve sort of basically gone from nothing, where we got given a basement by the Open University, and you know, see how you go to partnering with 160 universities from all over the world, and delivering education at scale.

Randy Silver: 

So and you keep saying the word we but that was when you started this about seven and a half years ago, and you’ve recently made a move, haven’t you?

Matt Walton: 

Yes. So yeah, so after seven and a half years, I decided it’s time to take a break and do something new. So I’m just kind of starting out on that path. So now is a really nice moment to kind of reflect upon that journey of, I guess, growing it over that kind of period. Because I think, you know, that’s some of the stuff that we’re going to be touching on tonight is how to do that.

Randy Silver: 

Yeah. So you grew from, as you said, from one of the first people in there to quite a sizable organisation. What did you learn about culture versus strategy over that point, because you said you were brought into credit strategy, but you’re also hiring some of the initial initial team and you grew it? What did you learn about that?

Matt Walton: 

So I think there’s like the what, what happened at the beginning, and then what happened next. So I think that very first piece was super important in terms of laying the foundations. So the early work that I did was really around persuading the EU to allow us to set up an organisation that worked in a way that was totally alien, from how they, you know, like, going back a few years as well, this sort of idea of working in a very kind of agile, cross functional way, putting things on Amazon Web Services, all those kinds of things are things that were not familiar to the EU at the time. So and I remember the first conversation that I had with Martin bean, who was the Vice Chancellor of the University at the time, and he basically asked me to promise him that I wouldn’t write a line of code, because they basically some of the use existing technology. And so the first one was really sort of saying, if you want to do something new and disruptive, then you know, using technology that’s nearly a decade old, and so on, may not be the right way to do it. So basically, it was about setting out like, the idea of a product vision, like the kind of principles of creating an autonomous team with a product manager, and that team kind of empowered to deliver upon the vision in whatever way they saw fit. But the key thing really was then about, like I trotted out to Milton Keynes, where the AAU was based every fortnight. And so we did our sprint review internally, and then it was about going out to them, and essentially bringing them on the journey and showing them this different way of how you could work. And I think it also made, the kind of stakeholders, they’re really fulfilled, bought into the process, because they got to kind of feed in in those kind of fortnightly moments, but they got to see it, and be part of that journey. So that that was that was one of the kind of key things and I think so setting out that kind of, I think in the in the talk that Randy mentioned early I kind of talked about, like all good bands have a manifesto. So we kind of have our manifesto is like this is this is how we’re gonna do it. So that was really important early on. And then the first members of that band are really important as well, because they kind of set the tone. So those first hires, I think, are the first people that you bring into work alongside you. And having a kind of shared set of values was really important. And then I think it’s about how you then consciously create the culture. And I think we’re gonna get on to talking about things like retrospectives, and so on. But like, we regularly created that space and that kind of fortnightly moment to reflect on how we were working. And at the beginning, it was very much about, you know, agreeing how we wanted to do that. And so I guess we made quite big decisions in some of those kinds of conversations, but it was about, you know, regularly making sure that we’re consciously doing that. And then I think, from that point, it’s been about there was a, I went to the South by Southwest festival about a year or so in like, and it was the point where we’ve gone from, I guess, being a ragbag bunch of freelancers that was doing a project to kind of going, how are we going to turn this into a sustainable thing? And I saw a talk by Phil Libin, who was the founder and CEO of Evernote. And he said something that really kind of resonated with me at the time and I guess has kind of been in the back of my mind ever since. In, which is that he said, My job is not to preserve culture is to make sure that it evolves in the right way. And that’s basically kind of how I thought about it since really is it’s about, you know, you’ve got to keep changing, if you’re working for a fast moving style that’s growing fast. Like, it’s never gonna like work when you’re bigger, but doing it in the right ways, the kind of trick.

Lily Smith: 

So one of the and one of the tools that you use for developing and evolving that culture in the right way is, is the is the retrospective. What’s your like? How do you approach retros? You know, how are they structured within the business? So within FutureLearn.

Matt Walton: 

So I guess just for a quick bit of context is we worked in a very kind of cross functional way, which, like, you know, obviously, many organisations do. But we had product teams that were generally about seven people and so on. And then we obviously then we line managed by discipline, so I was responsible originally for Product Management. And then we had a CTO responsible for technology and a creative director responsible for design. latterly, I did kind of step up and represent the whole organisation. But you know, broadly, that was the kind of pattern. So we used retrospectives in a variety of different ways. So I guess when we when we were small, we were one team. And that was the kind of whole organisation having at the end of we worked in fortnightly sprints. And at the end of that, fortnightly sprint, we would run the retrospective. Then as we grew, we kept that part of the culture. So each of the kinds of teams. So when we split into multiple teams, each of the teams will continue to do their kind of fortnightly retrospectives. But we did then start to do things like a discipline retrospective that happened less regularly. But the product management team would also meet and have their own kind of retrospective. We also had project retrospectives, which were more ad hoc. So if we had done something kind of big, we would involve everyone from across the business that was involved in that thing, to reflect on that project. And I guess the other kind of thing that we did that was very similar to retrospective was we introduced personal retrospectives. So we we actually use a tool called 15. Five to do it. But essentially, it meant that everyone, at the end of each week kind of finished the week by thinking about the things that have gone well, you know, what they might do differently and what their objectives for next week might be? So I think we have those, you know, there’s different kinds of moments of reflection at those kinds of different levels and different cadences.

Lily Smith: 

And how if you’ve got the kind of everyone’s familiar with, or, you know, would be familiar with how you would do it product team retrospective? Is it the same sort of format, then with the kind of, you know, what they did last week? What I’m planning next week, or, or month, or whatever it is? And then any blockers? Is it that kind of thing? Or did you change it in any way?

Matt Walton: 

So with with the discipline retrospectives? Yeah, very similar kind of drill. We actually, were you used Trello. And people could populate the board throughout the week and those kinds of things. So yeah, we did, we did very similar kinds of things to what you might do within a product team. Retrospective. But the tone was a bit different. Because it what was really nice about them as it was, when you spend, I guess most of your time, as a product manager with your cross functional team, rather than other product managers. Alongside like, the regular team meetings and so on, it was a way of creating that sense of team and helping, you know, reflect on our practices as a product management team, but also kind of, note the things across the organisation that were, you know, where it might be a consistent problem across teams. And that, I guess, for the team, it gave them a moment of silence kind of mutual support. And you know, often things would come up and an action would come out of it, and maybe to the PMS or going pair on like doing something about the problem. But actually lots of things that also came out might be things that I could take away to the kind of product leadership team, because they were bigger than something that maybe the team could fix themselves. And I think the other like for me as a leader it gave me because because one of the things that I you know, as as you kind of grow and one of the things that’s special about a retrospective is that it’s a safe place. And that means that actually as you grow probably, you know the the chief product officer being in a retrospective is not something that you want to happen. So I like for a long time I I’m not in a team retrospective because it needs to be a safe place. But it gave the team the discipline retrospective gave me the opportunity for some of those things that were themes that had been emerging, I guess within other retrospectives for them, to have them be back to me. And I read something yesterday, actually, which was a kind of reflection that as a leader, you quite often, when you step into that role, you start to feel that there’s a danger that you might lose some of the things that got you there in the first place. So being able to be empathetic, to have self awareness to be transparent. And you have to kind of give gratitude to your team, when you kind of do that difference. In your becoming a little more. Sometimes you stop doing all of those things, or because you’re just actually the team retros a really good way to stop doing that. Because you really do empathise with the problems that your team are going through. And you can do some of that kind of, I guess, servant leader thing of like, yeah, how can I? How can I make this better for all of you, because I’ve kind of, you know, heard and understood some of the things that are blocking you or whatever.

Lily Smith: 

So if you had lots of teams, all having retrospectives, how would you kind of get the summary of all of that information? would? Would it just be kind of fed up when it was necessary through kind of line management channels? Or, you know, would you sometimes join them? Or how would you do that.

Matt Walton: 

So generally, it would be through, I guess, a variety of sources. So through one to ones, through the discipline, retro that I mentioned, you know, if big things came up in the retro then the team would, would raise that. And I guess it depends upon what the problem might be, whether they would bring it to the whole product leadership team, or they bring it with me, or whether it will be in engineering. So I think part of the what we always tried to do is basically empower the team to make the changes themselves. And, and I think that’s kind of really, that’s a really key part of it. And that may be that they can do it themselves. Or it may be that they actually have to kind of go and seek help in some way. But it’s not for the same thing, really.

Randy Silver: 

You said something earlier about retros needs to be a safe space for people. I’m curious, if it’s our safe space, rituals can actually be a bit harmful. I’ve heard of stories where people get very accusatory at each other, and it’s not a good situation. So how do you run a good retro? What are the signs to watch out for that? Either you’ve set it up badly, or it’s going off the rails?

Matt Walton: 

Sure. So the key thing I think, is we always started our retros by reminding people of the prime directive. And, you know, we didn’t necessarily always do that, every team retro because teams, you know, get into the flow of doing it, but you know, quite regularly, like, you know, stepping back and reminding people of the Prime Directive, which was coined by I can’t remember his name, normally someone or other, but essentially basically says, regardless of what we discover, we assume that everyone was doing their best no matter, you know, with the resources and skills, I’m available, essentially. So essentially, is saying, basically reminding people to do that. I think facilitation is also quite key. So what we’d we try and do is to get somebody that wasn’t in the team being the kind of facilitator. And we actually have a retro bought on Slack that was about how a way of pairing or finding someone to, to facilitate your retrospective. And because every team needed someone to do that, like, that generally worked in terms of like, you know, someone saying, Yeah, okay, I’ll do that. So that’s very helpful. I think spending that kind of first bit of time, quietly, writing posts as well, in order to make sure that maybe they get the same voice is dominating. And then to do the kind of how you group together similar themes together, is also really helpful. And I think also, the key thing is really about arriving at some actions to do about it. So like, it shouldn’t be just a kind of gripe fest. And I think you always need to make sure that you do always have the what went well, what when this role, kind of to make sure that your, your imbalance there. So I think I think all of those things, and I think, you know, making sure that the team, it feels like a team and it’s about you know, how you collectively improve like it’s I think it’s always about like, how do you improve it’s not about like, yes, seeking blame, like that’s the thing it you just have to keep making sure that people are reminded of it.

Randy Silver: 

And for anyone who hasn’t seen it, I’ve managed to Find it while you were while you were talking that so it’s norm Kurth and the retrospective prime directive is, regardless of what we discover, we understand and truly believe that everyone did the best job that they could give them what they knew at the time, their skills and abilities, the resources available and the situation at hand.

Lily Smith: 

Randy’s dulcet tones.

Matt Walton: 

And yeah, we had, like, often like on meeting rewards as well. So you know, again, it’s like something starting to go off track, you know, someone might just tap the sign on the wall or whatever.

Lily Smith: 

So it sounds like an awful lot of retros Did you ever get retro fatigue and any sort of personalities in the team going, I’m fed up of thinking about what I’ve done or, you know, analysing it, I just want to, like, move forward.

Matt Walton: 

So I think the key thing is keep them short and sweet, change up the format, right, you don’t just have to do the, you know, the kind of smiley faces or whatever, like, I remember, you know, do things like the reserve, like what pulls us up and what brings us down, and it was like a kind of ship kind of thing. And this, you know, some of the same questions, but you can kind of mix up exactly how you do that, just to kind of keep it a bit interesting. And that’s also where if you’re switching around facilitators, and so on, that can kind of help them, you know, want to bring a bit more creativity to how that session might go. And so those and the I don’t know if I mentioned that when I gave the previous answer, but like, when we did the discipline, retros, they were only like, once a month, so we wouldn’t do those in Project retros would be, you know, fairly irregular, because it would be generally when there was a kind of bigger project that you kind of wanted to look back on. And the format. wanted to briefly talk through how we might run a

Randy Silver: 

before you do just quickly, I’ve seen examples of project retros, where they’re done right at right after launch, or sometimes like six months later, when would you hold them?

Matt Walton: 

It would depend I think, upon the project, you know, it’s a classic, it depends question. And it would also inevitably depend on people’s diaries as well. But I think you’d kind of want to do it soon enough to be able to, to make sure it’s fresh in people’s minds. So whatever you could do, really to do it relatively quick. But I think you know, there is that kind of like, when have we actually, when is it done, and generally things are never done. So things about picking the right moment where you think you can, you know, tease out the learnings from it. And the way we would generally do it is we would construct some kind of timeline like, and that would normally be crowd sourced before the session happened. And then the first part of the session might be people can then add their own recollections to it. And so what you start with is an agreed set of facts. And another fun thing you can do, and I guess this is like, you know, how do you mix up the watersheds, if at all, you can get people to draw a line of like how they felt during parts of the project. And that’s quite interesting, because like, if you’ve got people drawing lines, where you know, hi, as I’m feeling really good, and feeling really bad, you get all these wiggly lines, and sometimes everyone’s lines will be in exactly the same place. And you’ll be like, Why was that such an awesome moment or one thing on there? Why were we all like playfully wanting to dance. And sometimes guys will be in totally different places. And again, that’s also really interesting as to like, why, you know, from a design point of view, why were we feeling really upbeat, and yet the kinds of stakeholders or the engineers are having a totally different kind of perspective of that part of the project. And then it also then allows you to kind of like if you then get people to do the post it note thing. It allows you to then kind of group the themes chronologically sent music, you can do that quite quickly. And they can talk through it in a kind of chronological order. And it’s generally quite a cathartic thing, I think. And like if you can also involve, like, you know, we’ve done it, where we try and involve, like key members of the leadership team, like particularly if something went particularly well, particularly bad, has generally like people’s perspective of projects might be a little bit kind of distorted, depending on who they’ve heard, what from and so on. And I think it’s a really good way of getting a much more complete picture, particularly if you’ve set it out with that kind of blameless kind of premise. And I think if you then want to really do something about it, you kind of need leadership involved in that conversation, in order to, you know, maybe take on board some of the actions or things that need to change. That is quite a, you know, sometimes a trick to persuade people to kind of give up the time to do it. But I think, you know, if you can do it, then often people come away. And again, okay, I, you know, I’ve got a perspective on that I would never have done in another way. So, and if not, then I guess I guess it’s like, you know, how do you kind of share the kind of key takeaways in a way that’s quite kind of, I guess, first for all unreal or ease and so on?

Lily Smith: 

So did you ever find that people would wait until you know that if there was an issue kind of mid sprint or kind of mid project or, you know, with within a discipline, that people wouldn’t address it outside of the retro that the retro became becomes like the area where all problems or issues are aired? Or was that not the case?

Matt Walton: 

I don’t think so.

Lily Smith: 

I find it I find it interesting, because I’ve certainly been in situations before where something’s not gone quite right on a project or, you know, within a iteration of something. And someone will say to me, let’s have a retro, like, we literally just having a retro so that we can talk about this one thing that went wrong, but just seems really the wrong way round to do it.

Matt Walton: 

Yeah, I think, I mean, I think that’s where the cadence actually really helps. Because if you do it as a regular thing, then it’s not know what people don’t go into it, like kind of it is a big deal. Like, you know, like, sometimes that does happen with Project ones, like, you know, if a project went off the rails or whatever, then, you know, people may dread the retro. But if you’re with the kind of, if you can do it with a regularity, then it kind of means that it’s not something that people dread. I think the other thing that’s really good about sticking to a rhythm, is I think it makes people feel more willing to change because it means that you can try something for a fortnight, and then if it doesn’t work, then you can change, change it back again, then you’ve got that kind of moment to kind of do that. So it’s a very good way of like doing a kind of low risk rollout, because you can roll the change back.

Lily Smith: 

And do you have any rules around the actions that come out of retros, because sometimes you can end up with a great big long list.

Matt Walton: 

That’s the main rule. Think basically, a small list of management actions is basically exactly like sprint goals. If you’ve got too many sprint goals, you’re not going to achieve them all. And you’re going to end up feeling disappointed in yourself. So I think it’s about Yeah, making sure you’ve got not too many. And like, if they’re, you know, really important, they’re going to come up again. So, you know, don’t worry about it, and get making sure that someone’s responsibility as well, because I think the danger is that, you know, they can fall between the cracks. And you’ll end up in the next rhetoric and again, or why did nothing happen about this, but if it’s someone’s initials against it, and then potentially get more done and other things. Although you may find that people are scrambling around the last day of the sprint, because it’s like

Randy Silver: 

that’s okay, if it’s only a couple of weeks, that’s kind of the way Sprint’s are supposed to go. But I’m curious if FutureLearn evolved quite a lot over the seven and a half years that you were there. And you said, you’ve been doing the discipline retros for a long, big chunk of that, how did that change over time? And what did you how did the team evolve? What’s something that you might have learned from one of those?

Matt Walton: 

So I guess the first thing is, like, as we grew, where we might have changed how we approach those different retros. So one of the I guess, the kind of moments of growth was the point where we actually introduced the discipline retro, which was that the point where you had enough product managers for that to, you know, be a thing. And I think we’re actually, that meant to be that kind of feedback loop with me as kind of a team lead as well. So that was one one kind of part of it. We then I guess, evolved to the point where we ended up in, you know, as we got to a certain scale in that kind of classic quarterly planning cycle. And that cycle then also became a kind of nice way that we another cadence where you could make more organisational changes. So through the fortnightly retros, that’s where you would empower a team to make changes as much as they could, or, you know, bring things to leadership where you could potentially change some things on the fly. But certain things might be things that, you know, would need a change across the whole org. And so that kind of quarterly planning cycle moment was a moment where we might introduce things like that and we actually used so one of the things that emerged also probably through those kinds of conversations is we introduced the idea of a fire break sprint. It became a natural kind of segue between quarters and that then also be the place where we might do more maintaining modernise work from a technology point of view, but also became where we, you know, brought our eyes up to do more planning, but also where we might make a change to how we kind of might go about some of the processes and so on. So there’s a kind of natural moment to do it. Guess in terms of some of the things that came out of the discipline retro, like, you know, there’s a whole range of things. So changing from we use Pivotal Tracker for a long while, we switch to Trello, in terms of the kind of main tool and, and one team tried it, and then, you know, through that kind of feedback loop, you know, kind of other teams and move through it, we decided that product manager pairing was quite a good idea, because there’s been various things that came up as like this went well, this week, like, and it kind of made other PMS realise, actually, you know, in the same way that software engineers might spend time pairing on a problem as their product managers, that’s quite a good thing to do as well. When we went when we were going for investment. It was, you know, quite a good kind of way of like, getting a temperature check of like how the team were feeling. And off the back of that. I think I arranged a sort of separate hopes and fears session as a kind of way of point of helping the team talk through the things that they were hopeful for and worried about. And actually, that made them feel more in control of like this big unknown of like a potential new investor investing a large amount of money, what would that mean to the organisation?

Lily Smith: 

Did you ever go to a retro where there wasn’t any improvement that was needed to be done? Where it was like, we did a great job? Well, on everyone.

Matt Walton: 

I think those kinds of repost do happen. And that’s a really important thing. I think. So it’s just, it’s a natural moment where the team wants to give themselves a pat on the back. You know, I think it’s rare. I think there’s always something more like, Oh, can we do something about this?

Lily Smith: 

Yeah, yeah.

Matt Walton: 

But I think yeah, happens, I wouldn’t worry about it. I think that’s a really good moment for the team to kind of go, you know, collective, high five and group hug. If we haven’t seen them, I think, you know,

Lily Smith: 

that moment. And did you ever run? And you kind of mentioned about how you use them. So broadly, across the business? Did you ever try running them as leadership team, or like, on your strategy or anything like that?

Matt Walton: 

Yes. And I guess, that, I guess, is where some of those reflections that I mentioned earlier around, like, if you want to kind of make a real kind of step change, or really build the understanding about some of the underlying things that you know, might need addressing, like doing it with, with the with the leadership team, or we did do a couple of which were just as a leadership team, although I think often, like, it doesn’t make sense to do it as a leadership team, if it was you, the team doing our thing. But I think if it’s, if it’s a kind of wider thing, which is a kind of project and actually involving, you know, others in the organisation is probably more helpful. But yeah, and I think that as a tool, they work within that situation. The other thing that we did, so in general, I think, if you’re going to make changes to your culture, much like if you’re going to change your product, that continuous deployment of change is always more sensible, because it’s low cost, you’re not changing everything all at once. So the context is known. So it’s not like in the same way that you have, when you deploy a big change to your website, and you have a load of users going, Oh, I can’t know, I don’t know how this works anymore. I don’t know where the thing is, I hate it. Like, the same thing happens if you kind of make lots of changes to how you work. You need to do more columns, etc, etc, etc. So I think if you if you can deploy change regularly and in small ways, and where you can roll back changes, if it doesn’t work, or you can a be tested by this team trying it and then it’s adopted by the rest of the organisation, then that’s a good thing. There are some times in the same way that the come moments with your product where you need the change to be noticed. Like it needs to be a kind of step change, you’re going for the planet 10x thing, it’s time to do the redesign or, you know, pick other kinds of analogy here. Sometimes you do need to do that from a kind of culture and process point of view as well. So, and the retro can help you in that situation too. So, one thing that I did, at the point where, so we got this 50 million pounds of investment was clearly a kind of big thing, it led to quite a lot of change in the organisation. And so there was a smaller leadership team created. So I stepped up to rep to represent the whole product, whereas previously, I’d Co Co lead it with technology and design. And that created a moment where the, you know, very well respected leaders of those areas actually decided, now’s the time to go and do something different. So it created a moment where there was a lot of kind of, I guess, anxiety and worry within the design and technology organisations because, you know, suddenly, there’s a lot of change, and they don’t have leaders in those areas, or they’ve got leaders that are leaving. So the thing that I basically decided to do is like, how can we turn this moment of worry and change into a positive moment? And how can we kind of harness it? And so what I did then was, it sounds a bit BBC, but I did a thing that I called the Big retro. And essentially, it was about mixing up all of the all of the people from the product organisation and then doing five one hour sessions, which was about what are the things that you want to change about how we work that are bigger than your team? What are the things that we do well, that you want to preserve? What are the things that are really frustrating that we need to change? And what I do is do you have to do that, and they could be ideas that you’ve seen other organisations do. And it was a really low investment of time, because, you know, as come out through this conversation, everyone was very well drilled on how to do a retrospective. And lots of the things that came out of it were, you know, very familiar. So. So that was that was kind of reassuring, there were a few new things, or the word bits of emphasis that came out that were interesting. But it meant that any of the changes that we then made, the team felt bought into, because lovelies had that patient, many of the things that come out of that conversation, and many of the ideas that were proposed were things that we then put into action. Now I could have done that as a leader. And you know, it seemed like a kind of hunch. And like, you know, change is being done to us. There’s a great quote, I think about, you know, if change is done to you, it’s disturbing, but if it’s done by you, it’s exhilarating. That’s, I think, I think that was how that was a real matter of like turning into a real positive story, which is like, Well, how do we want to operate as a, as a product organisation and everyone kind of being part of that journey? As a way of using a retro, which was not about gradually making improvements. It was about like, you know, stepping back and kind of going, you know, what do we want to preserve and protect, and that what is really good about what we do? And what do we need to change. And it was really good, because it gave me a kind of reflection that you know, new leaders coming into the organisation. It was a kind of nice encapsulation of where we were at. And, you know, with the the exec team, as well like being able to have that conversation with some sort of very kind of real stories about you know, like, as a product or what we felt we did well and what we wanted to improve. Matt,

Lily Smith: 

thank you so much for coming on the podcast and talking to us. We’re like way over saving, but it’s been so interesting. Hearing all your stories about retros at FutureLearn. And some great tips in there for lots of people to take away. Thank you again.

Matt Walton: 

You’re very welcome. Thank you for having me.

Lily Smith: 

The product experience is the first and the best podcast from mine the product. Our hosts are me Lily Smith, and me Randy silver. Louron Pratt is our producer and Luke Smith is our editor.

Randy Silver: 

Our theme music is from Hamburg baseband power. That’s P AU. Thanks to Arnie killer who curates both product tank and MTP engage in Hamburg and who also plays bass in the band for letting us use their music. You can connect with your local product community via product tank regular free meetups in over 200 cities worldwide.

Lily Smith: 

If there’s not one near you, maybe you should think about starting one. To find out more go to mind the product.com forward slash product thank you

Matt Walton was one of the founders of FutureLearn, where he scaled the product team and organisation from nothing to the significant player in online education that it is today. Along the way, he learned a lot about continuous improvement of both the product and the processes for building it – including the use of four different types of retrospectives. He joins us on the podcast to level up our retro game.
       

Featured links

Follow Matt on LinkedIn and Twitter | Matt's website | Matt's Mind The Product talk 'Be in a Band, not an Orchestra: how to grow an Agile Product Team' | Funretrospective's 'The Retrospective Prime Directive' | Management tools at 15Five and Trello

Episode transcript

Randy Silver:  Lily, it's coming to the end of the quarter. You know what that means? It's time for a retro. Lily Smith:  Oh, I love retros. They can be a bit tricky, though, get it wrong. And it's a bit like Festivus from Seinfeld with the traditional airing of the grievances. Randy Silver:  I definitely been in a few of those. But we can do better than that. Right? You What kind of retro Should we run? Lily Smith:  You mean there are different kinds of fruit trees? Randy Silver:  Yeah. And I know that was a rhetorical question is, this week we're revisiting our chat with Matt Walton about the four types of retros that he uses. Lily Smith:  Randy so right through my cutting ploy. But now that we've set this up, let's hand the mic over to Matt. The product experience is brought to you by mind the product. Randy Silver:  Every week, we talk to the best product people from around the globe about how we can improve our practice, and build products that people love. Lily Smith:  Because it mind the product.com to catch up on past episodes and to discover an extensive library of great content and videos. Randy Silver:  Browse for free, will become a mind that product member to unlock premium articles, unseen videos, AMA's roundtables, discount store conferences around the world training opportunities for Lily Smith:  mining products also offers free product tank meetups in more than 200 cities. And there's probably Well Randy Silver:  Matt, thank you so much for joining us on the podcast tonight. We're really excited to have you. First time I met you was actually the first time I curated a product tank. And you gave a great talk about orchestras and festivals. And I'm sure we'll get to that later. But for now, and for anyone who wasn't there or hasn't watched the talk. Tell us a little bit about yourself. Who are you? And how did you get into product? Matt Walton:  Thanks, Randy. Really nice to be here tonight. I've been very much looking forward to spending some time in both of your companies around the sent me regularly. Have you got something to rant about. And I got back in touch and said I've got something to rant about. So me, Matt. So I've been doing I guess kind of product. for about 20 years or so I started my career at the BBC where they called everyone a producer. But I found myself basically working with software developers and designers to build new things at a time where it was kind of quite hands on and you were writing bits of HTML, yourself and so on. And so spent about 11 years at the BBC first on the public service side and then later working for the commercial arm. And whilst I was at the commercial arm, I got involved with this Meetup group called product tank. days when it was in a pub, and most of the conversation was about like, how do you explain to everyone in your organisation exactly what a product manager is, you know, as much of self help group, but I kind of felt, you know, I'd found my tribe. And so so that was when I, you know, first was termed a product manager and felt that, you know, finally someone had put a label on what it was that I did. And then after I left the BBC, I decided to kind of freelance for a bit and work for various different organisations. And whilst I was doing that, I stumbled into this amazing idea of a thing called FutureLearn. So for those of you that don't know what FutureLearn was, we were born out of the MOOC movement. So MOOCs are massive, open online courses. And it's when, back in 2012, there was a lot of American University started dabbling with putting courses online. And so the, the Open University, which is an amazing precious institution that was born in the 1960s, to basically take education to more people than I've ever had it before, through the mediums of TV and the Postal Service, decided that this movement was quite an interesting thing that maybe they should get in on the act on. So I got brought in, in the very kind of early days when a guy called Simon Nelson had been hired to be CEO of this startup, and he was someone that I'd worked with at the BBC. So I got brought in to think about what a vision for the product might be, and put together a plan for how I might deliver it. And before I knew it, I was hiring some software engineers building a prototype and committed to launching an MVP in September. We've I've now grown the product team. So the organisation is about 200 people, but the product is about 75, including technology design and product management. And we've now delivered courses to over 14 million learners around the world that we've signed up over seven and a half year period, and about 18 months so ago, we got 15 million pounds of investment from the seed group or a bit like an Australian LinkedIn. And so that basically valued the business at 100 million pounds. So we've sort of basically gone from nothing, where we got given a basement by the Open University, and you know, see how you go to partnering with 160 universities from all over the world, and delivering education at scale. Randy Silver:  So and you keep saying the word we but that was when you started this about seven and a half years ago, and you've recently made a move, haven't you? Matt Walton:  Yes. So yeah, so after seven and a half years, I decided it's time to take a break and do something new. So I'm just kind of starting out on that path. So now is a really nice moment to kind of reflect upon that journey of, I guess, growing it over that kind of period. Because I think, you know, that's some of the stuff that we're going to be touching on tonight is how to do that. Randy Silver:  Yeah. So you grew from, as you said, from one of the first people in there to quite a sizable organisation. What did you learn about culture versus strategy over that point, because you said you were brought into credit strategy, but you're also hiring some of the initial initial team and you grew it? What did you learn about that? Matt Walton:  So I think there's like the what, what happened at the beginning, and then what happened next. So I think that very first piece was super important in terms of laying the foundations. So the early work that I did was really around persuading the EU to allow us to set up an organisation that worked in a way that was totally alien, from how they, you know, like, going back a few years as well, this sort of idea of working in a very kind of agile, cross functional way, putting things on Amazon Web Services, all those kinds of things are things that were not familiar to the EU at the time. So and I remember the first conversation that I had with Martin bean, who was the Vice Chancellor of the University at the time, and he basically asked me to promise him that I wouldn't write a line of code, because they basically some of the use existing technology. And so the first one was really sort of saying, if you want to do something new and disruptive, then you know, using technology that's nearly a decade old, and so on, may not be the right way to do it. So basically, it was about setting out like, the idea of a product vision, like the kind of principles of creating an autonomous team with a product manager, and that team kind of empowered to deliver upon the vision in whatever way they saw fit. But the key thing really was then about, like I trotted out to Milton Keynes, where the AAU was based every fortnight. And so we did our sprint review internally, and then it was about going out to them, and essentially bringing them on the journey and showing them this different way of how you could work. And I think it also made, the kind of stakeholders, they're really fulfilled, bought into the process, because they got to kind of feed in in those kind of fortnightly moments, but they got to see it, and be part of that journey. So that that was that was one of the kind of key things and I think so setting out that kind of, I think in the in the talk that Randy mentioned early I kind of talked about, like all good bands have a manifesto. So we kind of have our manifesto is like this is this is how we're gonna do it. So that was really important early on. And then the first members of that band are really important as well, because they kind of set the tone. So those first hires, I think, are the first people that you bring into work alongside you. And having a kind of shared set of values was really important. And then I think it's about how you then consciously create the culture. And I think we're gonna get on to talking about things like retrospectives, and so on. But like, we regularly created that space and that kind of fortnightly moment to reflect on how we were working. And at the beginning, it was very much about, you know, agreeing how we wanted to do that. And so I guess we made quite big decisions in some of those kinds of conversations, but it was about, you know, regularly making sure that we're consciously doing that. And then I think, from that point, it's been about there was a, I went to the South by Southwest festival about a year or so in like, and it was the point where we've gone from, I guess, being a ragbag bunch of freelancers that was doing a project to kind of going, how are we going to turn this into a sustainable thing? And I saw a talk by Phil Libin, who was the founder and CEO of Evernote. And he said something that really kind of resonated with me at the time and I guess has kind of been in the back of my mind ever since. In, which is that he said, My job is not to preserve culture is to make sure that it evolves in the right way. And that's basically kind of how I thought about it since really is it's about, you know, you've got to keep changing, if you're working for a fast moving style that's growing fast. Like, it's never gonna like work when you're bigger, but doing it in the right ways, the kind of trick. Lily Smith:  So one of the and one of the tools that you use for developing and evolving that culture in the right way is, is the is the retrospective. What's your like? How do you approach retros? You know, how are they structured within the business? So within FutureLearn. Matt Walton:  So I guess just for a quick bit of context is we worked in a very kind of cross functional way, which, like, you know, obviously, many organisations do. But we had product teams that were generally about seven people and so on. And then we obviously then we line managed by discipline, so I was responsible originally for Product Management. And then we had a CTO responsible for technology and a creative director responsible for design. latterly, I did kind of step up and represent the whole organisation. But you know, broadly, that was the kind of pattern. So we used retrospectives in a variety of different ways. So I guess when we when we were small, we were one team. And that was the kind of whole organisation having at the end of we worked in fortnightly sprints. And at the end of that, fortnightly sprint, we would run the retrospective. Then as we grew, we kept that part of the culture. So each of the kinds of teams. So when we split into multiple teams, each of the teams will continue to do their kind of fortnightly retrospectives. But we did then start to do things like a discipline retrospective that happened less regularly. But the product management team would also meet and have their own kind of retrospective. We also had project retrospectives, which were more ad hoc. So if we had done something kind of big, we would involve everyone from across the business that was involved in that thing, to reflect on that project. And I guess the other kind of thing that we did that was very similar to retrospective was we introduced personal retrospectives. So we we actually use a tool called 15. Five to do it. But essentially, it meant that everyone, at the end of each week kind of finished the week by thinking about the things that have gone well, you know, what they might do differently and what their objectives for next week might be? So I think we have those, you know, there's different kinds of moments of reflection at those kinds of different levels and different cadences. Lily Smith:  And how if you've got the kind of everyone's familiar with, or, you know, would be familiar with how you would do it product team retrospective? Is it the same sort of format, then with the kind of, you know, what they did last week? What I'm planning next week, or, or month, or whatever it is? And then any blockers? Is it that kind of thing? Or did you change it in any way? Matt Walton:  So with with the discipline retrospectives? Yeah, very similar kind of drill. We actually, were you used Trello. And people could populate the board throughout the week and those kinds of things. So yeah, we did, we did very similar kinds of things to what you might do within a product team. Retrospective. But the tone was a bit different. Because it what was really nice about them as it was, when you spend, I guess most of your time, as a product manager with your cross functional team, rather than other product managers. Alongside like, the regular team meetings and so on, it was a way of creating that sense of team and helping, you know, reflect on our practices as a product management team, but also kind of, note the things across the organisation that were, you know, where it might be a consistent problem across teams. And that, I guess, for the team, it gave them a moment of silence kind of mutual support. And you know, often things would come up and an action would come out of it, and maybe to the PMS or going pair on like doing something about the problem. But actually lots of things that also came out might be things that I could take away to the kind of product leadership team, because they were bigger than something that maybe the team could fix themselves. And I think the other like for me as a leader it gave me because because one of the things that I you know, as as you kind of grow and one of the things that's special about a retrospective is that it's a safe place. And that means that actually as you grow probably, you know the the chief product officer being in a retrospective is not something that you want to happen. So I like for a long time I I'm not in a team retrospective because it needs to be a safe place. But it gave the team the discipline retrospective gave me the opportunity for some of those things that were themes that had been emerging, I guess within other retrospectives for them, to have them be back to me. And I read something yesterday, actually, which was a kind of reflection that as a leader, you quite often, when you step into that role, you start to feel that there's a danger that you might lose some of the things that got you there in the first place. So being able to be empathetic, to have self awareness to be transparent. And you have to kind of give gratitude to your team, when you kind of do that difference. In your becoming a little more. Sometimes you stop doing all of those things, or because you're just actually the team retros a really good way to stop doing that. Because you really do empathise with the problems that your team are going through. And you can do some of that kind of, I guess, servant leader thing of like, yeah, how can I? How can I make this better for all of you, because I've kind of, you know, heard and understood some of the things that are blocking you or whatever. Lily Smith:  So if you had lots of teams, all having retrospectives, how would you kind of get the summary of all of that information? would? Would it just be kind of fed up when it was necessary through kind of line management channels? Or, you know, would you sometimes join them? Or how would you do that. Matt Walton:  So generally, it would be through, I guess, a variety of sources. So through one to ones, through the discipline, retro that I mentioned, you know, if big things came up in the retro then the team would, would raise that. And I guess it depends upon what the problem might be, whether they would bring it to the whole product leadership team, or they bring it with me, or whether it will be in engineering. So I think part of the what we always tried to do is basically empower the team to make the changes themselves. And, and I think that's kind of really, that's a really key part of it. And that may be that they can do it themselves. Or it may be that they actually have to kind of go and seek help in some way. But it's not for the same thing, really. Randy Silver:  You said something earlier about retros needs to be a safe space for people. I'm curious, if it's our safe space, rituals can actually be a bit harmful. I've heard of stories where people get very accusatory at each other, and it's not a good situation. So how do you run a good retro? What are the signs to watch out for that? Either you've set it up badly, or it's going off the rails? Matt Walton:  Sure. So the key thing I think, is we always started our retros by reminding people of the prime directive. And, you know, we didn't necessarily always do that, every team retro because teams, you know, get into the flow of doing it, but you know, quite regularly, like, you know, stepping back and reminding people of the Prime Directive, which was coined by I can't remember his name, normally someone or other, but essentially basically says, regardless of what we discover, we assume that everyone was doing their best no matter, you know, with the resources and skills, I'm available, essentially. So essentially, is saying, basically reminding people to do that. I think facilitation is also quite key. So what we'd we try and do is to get somebody that wasn't in the team being the kind of facilitator. And we actually have a retro bought on Slack that was about how a way of pairing or finding someone to, to facilitate your retrospective. And because every team needed someone to do that, like, that generally worked in terms of like, you know, someone saying, Yeah, okay, I'll do that. So that's very helpful. I think spending that kind of first bit of time, quietly, writing posts as well, in order to make sure that maybe they get the same voice is dominating. And then to do the kind of how you group together similar themes together, is also really helpful. And I think also, the key thing is really about arriving at some actions to do about it. So like, it shouldn't be just a kind of gripe fest. And I think you always need to make sure that you do always have the what went well, what when this role, kind of to make sure that your, your imbalance there. So I think I think all of those things, and I think, you know, making sure that the team, it feels like a team and it's about you know, how you collectively improve like it's I think it's always about like, how do you improve it's not about like, yes, seeking blame, like that's the thing it you just have to keep making sure that people are reminded of it. Randy Silver:  And for anyone who hasn't seen it, I've managed to Find it while you were while you were talking that so it's norm Kurth and the retrospective prime directive is, regardless of what we discover, we understand and truly believe that everyone did the best job that they could give them what they knew at the time, their skills and abilities, the resources available and the situation at hand. Lily Smith:  Randy's dulcet tones. Matt Walton:  And yeah, we had, like, often like on meeting rewards as well. So you know, again, it's like something starting to go off track, you know, someone might just tap the sign on the wall or whatever. Lily Smith:  So it sounds like an awful lot of retros Did you ever get retro fatigue and any sort of personalities in the team going, I'm fed up of thinking about what I've done or, you know, analysing it, I just want to, like, move forward. Matt Walton:  So I think the key thing is keep them short and sweet, change up the format, right, you don't just have to do the, you know, the kind of smiley faces or whatever, like, I remember, you know, do things like the reserve, like what pulls us up and what brings us down, and it was like a kind of ship kind of thing. And this, you know, some of the same questions, but you can kind of mix up exactly how you do that, just to kind of keep it a bit interesting. And that's also where if you're switching around facilitators, and so on, that can kind of help them, you know, want to bring a bit more creativity to how that session might go. And so those and the I don't know if I mentioned that when I gave the previous answer, but like, when we did the discipline, retros, they were only like, once a month, so we wouldn't do those in Project retros would be, you know, fairly irregular, because it would be generally when there was a kind of bigger project that you kind of wanted to look back on. And the format. wanted to briefly talk through how we might run a Randy Silver:  before you do just quickly, I've seen examples of project retros, where they're done right at right after launch, or sometimes like six months later, when would you hold them? Matt Walton:  It would depend I think, upon the project, you know, it's a classic, it depends question. And it would also inevitably depend on people's diaries as well. But I think you'd kind of want to do it soon enough to be able to, to make sure it's fresh in people's minds. So whatever you could do, really to do it relatively quick. But I think you know, there is that kind of like, when have we actually, when is it done, and generally things are never done. So things about picking the right moment where you think you can, you know, tease out the learnings from it. And the way we would generally do it is we would construct some kind of timeline like, and that would normally be crowd sourced before the session happened. And then the first part of the session might be people can then add their own recollections to it. And so what you start with is an agreed set of facts. And another fun thing you can do, and I guess this is like, you know, how do you mix up the watersheds, if at all, you can get people to draw a line of like how they felt during parts of the project. And that's quite interesting, because like, if you've got people drawing lines, where you know, hi, as I'm feeling really good, and feeling really bad, you get all these wiggly lines, and sometimes everyone's lines will be in exactly the same place. And you'll be like, Why was that such an awesome moment or one thing on there? Why were we all like playfully wanting to dance. And sometimes guys will be in totally different places. And again, that's also really interesting as to like, why, you know, from a design point of view, why were we feeling really upbeat, and yet the kinds of stakeholders or the engineers are having a totally different kind of perspective of that part of the project. And then it also then allows you to kind of like if you then get people to do the post it note thing. It allows you to then kind of group the themes chronologically sent music, you can do that quite quickly. And they can talk through it in a kind of chronological order. And it's generally quite a cathartic thing, I think. And like if you can also involve, like, you know, we've done it, where we try and involve, like key members of the leadership team, like particularly if something went particularly well, particularly bad, has generally like people's perspective of projects might be a little bit kind of distorted, depending on who they've heard, what from and so on. And I think it's a really good way of getting a much more complete picture, particularly if you've set it out with that kind of blameless kind of premise. And I think if you then want to really do something about it, you kind of need leadership involved in that conversation, in order to, you know, maybe take on board some of the actions or things that need to change. That is quite a, you know, sometimes a trick to persuade people to kind of give up the time to do it. But I think, you know, if you can do it, then often people come away. And again, okay, I, you know, I've got a perspective on that I would never have done in another way. So, and if not, then I guess I guess it's like, you know, how do you kind of share the kind of key takeaways in a way that's quite kind of, I guess, first for all unreal or ease and so on? Lily Smith:  So did you ever find that people would wait until you know that if there was an issue kind of mid sprint or kind of mid project or, you know, with within a discipline, that people wouldn't address it outside of the retro that the retro became becomes like the area where all problems or issues are aired? Or was that not the case? Matt Walton:  I don't think so. Lily Smith:  I find it I find it interesting, because I've certainly been in situations before where something's not gone quite right on a project or, you know, within a iteration of something. And someone will say to me, let's have a retro, like, we literally just having a retro so that we can talk about this one thing that went wrong, but just seems really the wrong way round to do it. Matt Walton:  Yeah, I think, I mean, I think that's where the cadence actually really helps. Because if you do it as a regular thing, then it's not know what people don't go into it, like kind of it is a big deal. Like, you know, like, sometimes that does happen with Project ones, like, you know, if a project went off the rails or whatever, then, you know, people may dread the retro. But if you're with the kind of, if you can do it with a regularity, then it kind of means that it's not something that people dread. I think the other thing that's really good about sticking to a rhythm, is I think it makes people feel more willing to change because it means that you can try something for a fortnight, and then if it doesn't work, then you can change, change it back again, then you've got that kind of moment to kind of do that. So it's a very good way of like doing a kind of low risk rollout, because you can roll the change back. Lily Smith:  And do you have any rules around the actions that come out of retros, because sometimes you can end up with a great big long list. Matt Walton:  That's the main rule. Think basically, a small list of management actions is basically exactly like sprint goals. If you've got too many sprint goals, you're not going to achieve them all. And you're going to end up feeling disappointed in yourself. So I think it's about Yeah, making sure you've got not too many. And like, if they're, you know, really important, they're going to come up again. So, you know, don't worry about it, and get making sure that someone's responsibility as well, because I think the danger is that, you know, they can fall between the cracks. And you'll end up in the next rhetoric and again, or why did nothing happen about this, but if it's someone's initials against it, and then potentially get more done and other things. Although you may find that people are scrambling around the last day of the sprint, because it's like Randy Silver:  that's okay, if it's only a couple of weeks, that's kind of the way Sprint's are supposed to go. But I'm curious if FutureLearn evolved quite a lot over the seven and a half years that you were there. And you said, you've been doing the discipline retros for a long, big chunk of that, how did that change over time? And what did you how did the team evolve? What's something that you might have learned from one of those? Matt Walton:  So I guess the first thing is, like, as we grew, where we might have changed how we approach those different retros. So one of the I guess, the kind of moments of growth was the point where we actually introduced the discipline retro, which was that the point where you had enough product managers for that to, you know, be a thing. And I think we're actually, that meant to be that kind of feedback loop with me as kind of a team lead as well. So that was one one kind of part of it. We then I guess, evolved to the point where we ended up in, you know, as we got to a certain scale in that kind of classic quarterly planning cycle. And that cycle then also became a kind of nice way that we another cadence where you could make more organisational changes. So through the fortnightly retros, that's where you would empower a team to make changes as much as they could, or, you know, bring things to leadership where you could potentially change some things on the fly. But certain things might be things that, you know, would need a change across the whole org. And so that kind of quarterly planning cycle moment was a moment where we might introduce things like that and we actually used so one of the things that emerged also probably through those kinds of conversations is we introduced the idea of a fire break sprint. It became a natural kind of segue between quarters and that then also be the place where we might do more maintaining modernise work from a technology point of view, but also became where we, you know, brought our eyes up to do more planning, but also where we might make a change to how we kind of might go about some of the processes and so on. So there's a kind of natural moment to do it. Guess in terms of some of the things that came out of the discipline retro, like, you know, there's a whole range of things. So changing from we use Pivotal Tracker for a long while, we switch to Trello, in terms of the kind of main tool and, and one team tried it, and then, you know, through that kind of feedback loop, you know, kind of other teams and move through it, we decided that product manager pairing was quite a good idea, because there's been various things that came up as like this went well, this week, like, and it kind of made other PMS realise, actually, you know, in the same way that software engineers might spend time pairing on a problem as their product managers, that's quite a good thing to do as well. When we went when we were going for investment. It was, you know, quite a good kind of way of like, getting a temperature check of like how the team were feeling. And off the back of that. I think I arranged a sort of separate hopes and fears session as a kind of way of point of helping the team talk through the things that they were hopeful for and worried about. And actually, that made them feel more in control of like this big unknown of like a potential new investor investing a large amount of money, what would that mean to the organisation? Lily Smith:  Did you ever go to a retro where there wasn't any improvement that was needed to be done? Where it was like, we did a great job? Well, on everyone. Matt Walton:  I think those kinds of repost do happen. And that's a really important thing. I think. So it's just, it's a natural moment where the team wants to give themselves a pat on the back. You know, I think it's rare. I think there's always something more like, Oh, can we do something about this? Lily Smith:  Yeah, yeah. Matt Walton:  But I think yeah, happens, I wouldn't worry about it. I think that's a really good moment for the team to kind of go, you know, collective, high five and group hug. If we haven't seen them, I think, you know, Lily Smith:  that moment. And did you ever run? And you kind of mentioned about how you use them. So broadly, across the business? Did you ever try running them as leadership team, or like, on your strategy or anything like that? Matt Walton:  Yes. And I guess, that, I guess, is where some of those reflections that I mentioned earlier around, like, if you want to kind of make a real kind of step change, or really build the understanding about some of the underlying things that you know, might need addressing, like doing it with, with the with the leadership team, or we did do a couple of which were just as a leadership team, although I think often, like, it doesn't make sense to do it as a leadership team, if it was you, the team doing our thing. But I think if it's, if it's a kind of wider thing, which is a kind of project and actually involving, you know, others in the organisation is probably more helpful. But yeah, and I think that as a tool, they work within that situation. The other thing that we did, so in general, I think, if you're going to make changes to your culture, much like if you're going to change your product, that continuous deployment of change is always more sensible, because it's low cost, you're not changing everything all at once. So the context is known. So it's not like in the same way that you have, when you deploy a big change to your website, and you have a load of users going, Oh, I can't know, I don't know how this works anymore. I don't know where the thing is, I hate it. Like, the same thing happens if you kind of make lots of changes to how you work. You need to do more columns, etc, etc, etc. So I think if you if you can deploy change regularly and in small ways, and where you can roll back changes, if it doesn't work, or you can a be tested by this team trying it and then it's adopted by the rest of the organisation, then that's a good thing. There are some times in the same way that the come moments with your product where you need the change to be noticed. Like it needs to be a kind of step change, you're going for the planet 10x thing, it's time to do the redesign or, you know, pick other kinds of analogy here. Sometimes you do need to do that from a kind of culture and process point of view as well. So, and the retro can help you in that situation too. So, one thing that I did, at the point where, so we got this 50 million pounds of investment was clearly a kind of big thing, it led to quite a lot of change in the organisation. And so there was a smaller leadership team created. So I stepped up to rep to represent the whole product, whereas previously, I'd Co Co lead it with technology and design. And that created a moment where the, you know, very well respected leaders of those areas actually decided, now's the time to go and do something different. So it created a moment where there was a lot of kind of, I guess, anxiety and worry within the design and technology organisations because, you know, suddenly, there's a lot of change, and they don't have leaders in those areas, or they've got leaders that are leaving. So the thing that I basically decided to do is like, how can we turn this moment of worry and change into a positive moment? And how can we kind of harness it? And so what I did then was, it sounds a bit BBC, but I did a thing that I called the Big retro. And essentially, it was about mixing up all of the all of the people from the product organisation and then doing five one hour sessions, which was about what are the things that you want to change about how we work that are bigger than your team? What are the things that we do well, that you want to preserve? What are the things that are really frustrating that we need to change? And what I do is do you have to do that, and they could be ideas that you've seen other organisations do. And it was a really low investment of time, because, you know, as come out through this conversation, everyone was very well drilled on how to do a retrospective. And lots of the things that came out of it were, you know, very familiar. So. So that was that was kind of reassuring, there were a few new things, or the word bits of emphasis that came out that were interesting. But it meant that any of the changes that we then made, the team felt bought into, because lovelies had that patient, many of the things that come out of that conversation, and many of the ideas that were proposed were things that we then put into action. Now I could have done that as a leader. And you know, it seemed like a kind of hunch. And like, you know, change is being done to us. There's a great quote, I think about, you know, if change is done to you, it's disturbing, but if it's done by you, it's exhilarating. That's, I think, I think that was how that was a real matter of like turning into a real positive story, which is like, Well, how do we want to operate as a, as a product organisation and everyone kind of being part of that journey? As a way of using a retro, which was not about gradually making improvements. It was about like, you know, stepping back and kind of going, you know, what do we want to preserve and protect, and that what is really good about what we do? And what do we need to change. And it was really good, because it gave me a kind of reflection that you know, new leaders coming into the organisation. It was a kind of nice encapsulation of where we were at. And, you know, with the the exec team, as well like being able to have that conversation with some sort of very kind of real stories about you know, like, as a product or what we felt we did well and what we wanted to improve. Matt, Lily Smith:  thank you so much for coming on the podcast and talking to us. We're like way over saving, but it's been so interesting. Hearing all your stories about retros at FutureLearn. And some great tips in there for lots of people to take away. Thank you again. Matt Walton:  You're very welcome. Thank you for having me. Lily Smith:  The product experience is the first and the best podcast from mine the product. Our hosts are me Lily Smith, and me Randy silver. Louron Pratt is our producer and Luke Smith is our editor. Randy Silver:  Our theme music is from Hamburg baseband power. That's P AU. Thanks to Arnie killer who curates both product tank and MTP engage in Hamburg and who also plays bass in the band for letting us use their music. You can connect with your local product community via product tank regular free meetups in over 200 cities worldwide. Lily Smith:  If there's not one near you, maybe you should think about starting one. To find out more go to mind the product.com forward slash product thank you