So many missed that a profanity filter would prevent the
residents of Scunthorpe from creating an AOL account, that the use of
sat navs could ruin previously quiet streets for their residents, or that a social network for university students would end up
interfering with democracy. Building responsibly isn’t a headline topic for every product manager.
Mozilla and Harvard Fellow
Kathy Pham is one of a growing number of product leaders calling for this to be rethought and reprioritised. She says: “This idea of a responsible product hasn't been a main category in the same way as user experience, or product strategy, or product launch has always been. But we need to consider how we integrate responsible thinking into our roadmaps and into our product launches, into marketing or strategy or user experience so that we responsibly think about enough communities. We have a world where there are so many blind spots, even for people with the best intent who want to change and connect the world. We have so many different criteria and metrics to follow that sometimes we cause harm with a product that’s intended to do good.”
Her comments are echoed by others.
Roisi Proven, Director of Product at Altmetric, says that all too often ethical considerations are treated in a very similar way to accessibility work when building products: “You build everything else and then at the very end you think about the things that I would consider most important. Is it accessible? Is it fair? Is it unbiased? Is it going to harm anyone? Those questions are not asked throughout, only at the end.”
Mariah Hay, VP of Product at Pluralsight, in a keynote (
First, Do No Harm) at #mtpcon San Francisco in 2018, pointed out that the onus is on all of us as individuals to strive to build products in an ethical way. She said: “Companies aren’t inherently unethical, it’s just that they aren’t people. So ethics must live with you, the individual practitioner - it’s the doctor not the hospital that takes the Hippocratic Oath… it’s we problem finders and problem solvers who must decide not to be problem creators.”
And in his 2018 #mtpcon London talk,
Building Product in an Age of Distrust,
Ryan Freitas, Director of Product Design at Facebook, said: “I contend that builders of product live in a new age of distrust… what we can’t ignore is that previously neutral, sometimes positive, even cheerleading responses to new product releases or even feature updates are gone. We now have to make certain that everything we do is grounded in truth, positive metrics, and the message that we are living up the expectations that we’ve set.”
Why Don’t we Build Responsibly?
As humans, most of us want to do the right thing so why aren’t ethical and social responsibilities an unvarying constant in a product’s development, particularly since product people generally strive to operate in an environment of psychological safety? Part of the problem, Roisi thinks, is that ethics is not something you can tick off a list, rather it has to be continuous. “We're working on articulating each bias that we have, and then addressing it in a systematic way. And really, you have to constantly be finding your biases,” she says. Kathy adds that she finds this failure to address whether a product is responsible and ethical quite surprising because “product managers love to take credit for things. We’re the ones on stage talking about our products, talking about taking responsibility, we even say no one can fail, yet no one takes responsibility for this. Part of it, I think, is profit misalignment.”
It’s a valid point. Kathy says that startups - of necessity - are typically mission focused till they get to a point where they have to find people to either fund them or use their product: “And then it becomes hyper focused on that. It's very hard to find that balance of being profitable, and having people use the product, and being responsible.”
A Different Focus
Mind the Product co-founder Martin Eriksson is also a Product Partner at venture capital firm
EQT Ventures. He comments that it’s not that VCs are inherently unethical, but they are focused on their return and exit strategy because - as we often forget - they have to answer to their investors. It’s a funding model that’s not going to change so product people need to show that building ethically drives long-term value. Martin says: “It’s up to the CPO or product director to put context around growth numbers and to think about ethics in terms of the bigger picture and the company’s long-term goals. This thinking needs to be expressed in the right language for investors so that they can see that building responsibly is better for customers, the market, and their long-term value.”
While it's
not perfect, Apple is an example of a successful big tech business that is now putting user privacy first and actively limiting how companies track their users on the Apple platform, escalating
to a public confrontation with ad-supported businesses like Google and Facebook.
It’s easy to zero in on the ethical shortcomings of tech behemoths like Facebook, but for most businesses the discussion about how to build responsibly is naturally very different. Martin points to Mariah Hay’s presentation at #mtpcon San Francisco in 2018,
First, Do No Harm mentioned earlier, as a relatable example of an ethics case study to learn from. He also points out that it’s important to remember that the choices you make may have a long-term impact. Exploring different business models then is a valuable exercise for young companies: “Inevitably, Facebook's business model has driven them to be all about traffic and click engagement, which leads them to promote controversial behaviour, whereas if they had started by charging people $5 a month to use Facebook, they would have made different decisions.”
Time to Think Differently?
Kathy believes that simple ignorance is another reason why building responsibly gets overlooked, despite the product craft being full of smart and passionate people. She says: “In 2018, if you had asked someone about the political climate and state of technology in Myanmar, they would probably have had no idea.” Facebook has admitted that
Facebook and WhatsApp incited violence against the Rohingya in Myanmar which ultimately led to genocide, because its employees lacked an understanding of both the political climate and the way technology use was growing in the country. In Myanmar at the time, internet use was growing, and was often limited to certain apps, so for many Facebook was the news.
[caption id="attachment_23121" align="alignnone" width="1024"]
Facebook was used to incite violence against the Rohingya people in Myanmar (Image: Shutterstock)[/caption]
Kathy thinks that the way that computer science is taught in universities and the engineering culture it spawns is partly responsible for this ignorance and lack of understanding. Students learn to build technology really well, she says, but the social science and ethics involved in building tech gets overlooked. “It’s led to a culture where you learn to build a piece of technology and how to get a ton of people to use it, and then maybe worry later about how they use it.” Designer and futurist
Cennydd Bowles, who focuses on responsible design, adds that none of his work comes from small companies. “They think it’s a luxury, it’s icing on the cake,” he says.
Kathy teaches a product management class at Harvard and says that thankfully she now sees a greater appreciation among her students of how products can be misused. Unfortunately, however, as they become focused on their careers it seems that responsible product development isn’t the overarching priority it should be. For example, she also helps to run the
Responsible Computer Science Challenge, a $3.5 million initiative to integrate ethics into undergraduate computer science curricula across the US. She says: “I hear that when we teach these topics the students are super-engaged in class. That is really promising. Then when it comes to final projects, sometimes those concepts do not stick.” And last year, for instance, she was involved in coaching startups in an incubator called
Fix the Internet. She says: “On one hand, it was exciting to see so many people want to fix the internet. But as teams were working to create their minimum viable products and grow their companies, it sometimes was hard to prioritise values or early moderation over just launching and trying to stay afloat.”
A Product Manager’s Role
Kathy comments that the above serves to underline the importance of the product manager’s role in delivering responsibly-built products. “Product managers have such a significant impact,” she says, “whether it's our daily stand ups, how we write our requirements, even how we talk about what our minimum viable product is like.” Product managers need to be the ones questioning whether a product is ready to ship, whether all the relevant studies have been done, and the right questions answered. “Engineering and User Experience can be so focused on building and understanding the user experience; designers can have so much power over how a product looks and operates,” says Kathy. “Product managers need to be the ones who say, ‘Hey, we're not going to launch until we make sure we do XYZ’.”
Cennydd thinks that, driven by consumer pressure, the build responsibly message is beginning to get through to businesses, adding “the two things that will really push this are case studies and regulation. We need a diverse body of case studies we can draw on”. He points to Salesforce, Microsoft and Google as tech businesses that are taking a lead for others to follow. Salesforce in particular is building a culture of responsible technology and has done extensive research on consumer attitudes to
ethics, human rights and technology. And, as Roisi Proven points out, you only have to look at the
recent growth of messaging app Signal at the expense of WhatsApp, to see that there can be huge dividends in ethical product development. If you have any experiences or case study material that would help to build a bank of resources on ethical product development, please get in touch. Email
editor@mindtheproduct.com.
Regulation may be some way off in the future, but the beginnings are already with us. The EU, for example, already has
Ethics Guidelines for Trustworthy Artificial Intelligence, and at the time of writing
dating app Grindr is potentially in line for a $12m fine under GDPR regulations in relation to consent violations in the sharing of its users’ personal data. This article,
The Regulator’s Puzzle, by tech and media analyst Ben Evans looks at what a complex and multi-faceted topic tech regulation is.
Some Practical Guidance
What guidance then is currently available? And what steps can you take to help your products get built in a responsible way? All product managers love a framework but as yet, sadly, those that address this area are few and far between. Instead, Kathy, Cennydd, and Roisi are here to help with some simple guidance, tips, and recommended tools.
Think Through Intended and Unintended Consequences
As Kathy says, you can start by thinking through both the intended and unintended consequences of how people might use your product. Cennydd Bowles paraphrases science and technology philosopher
Don Ihde when he says that technology is multistable - meaning that technologies can be interpreted in lots of different ways: “That multistability means whatever you plan for, it's also going to have a bunch of alternative uses, so you have to try to anticipate them.”
Cennydd uses some tried and tested techniques to do this. One is a risk mapping exercise, which sorts potential risk or ethical harm resulting from a piece of work into categories and then to assess their possible impact from there. “A lot of my work is this sort of anticipatory analysis,” he says, “and you can use practices like speculative design and design fiction to tease out some of those ideas.”
Consider the Needs of Hidden Stakeholders
Cennydd’s work also looks at how businesses might reframe their responsibilities. “That means looking for hidden stakeholders because the people harmed by these technologies frequently aren't users.” AirBNB is a good example here he says - the people who are really affected by AirBNB are neighbours and local communities, because their rents go up and neighborhoods get hollowed out. He uses the Actor Triangle from NordKapp’s
Actionable Futures Toolkit to draw out who else might be affected by product decisions.
[caption id="attachment_23120" align="aligncenter" width="1024"]
Hidden stakeholders: the people really affected by AirBNB are neighbours (Image: Shutterstock)[/caption]
Next, you need to get an understanding of these hidden stakeholders’ needs. Cennydd says: “This is where you use things like co-design, participatory design, and you listen to activists from those communities. You consult with them about any possible impacts and how you can mitigate them. So this tiny square of design actually becomes quite big.” If this sounds expensive in terms of time and money then Cennydd points out that it’s important to prioritise and perform impact risk assessment on all these hidden stakeholders. “You need to be able to recognise what’s within your scope and where the most severe harms lie. At some point you’ll say this is a social problem that I can only marginally influence so I can’t commit a team to it. It’s a three-step process, so you anticipate, evaluate and then react to those opportunities or problems.”
Focus on Design
Good product managers should view design as integral to their product, adds Kathy, and she points to equitable design frameworks that product managers can make use of when figuring out the issues in building responsibly. She mentions, for example, the
EquityXDesign framework, which looks to “merge the consciousness of racial equity work with the methodology of design thinking”.
Be Diverse
Much has been written about how diversity in teams rightly addresses many of these issues, but as Kathy has already pointed out, this doesn’t always happen. “Even if you don’t have a diverse team, you need to get those voices in the room and treat them as equal to others,” she says. She say
Doteveryone’s Consequence Scanning framework can be a useful tool and also suggests that product managers build in - say once a week - time to think through ways they are having an impact on society, and that they take time to bring in subject-matter experts during the product development cycle. She mentions as an example that left-handed people found it
harder to take photos with an Android phone - because there were no left-handers on the development team so no one considered it. She adds: “Whether it’s your minimum viable product, your product launch, your product roadmap, just try adding these little bits to your product cycle and see what happens.” In a similar vein Cennydd also suggests that product managers try to broaden their pool of participants in user testing.
[caption id="attachment_23123" align="aligncenter" width="1024"]
Diversity in teams helps to address many of the issues in responsible thinking (Image: Shutterstock)[/caption]
Use a Check Metric
Naturally enough metrics are another area where building responsibly can be monitored. Cennydd strongly recommends that product managers pick a metric that balances each of their metrics - sometimes referred to as a check metric. "If you pick just one metric everyone will try to game it. So you need to pick a counterbalancing metric. Take time to market, if you can ship more quickly by cutting corners then a balancing metric could be your app store rating. If you ship rubbish you’ll see it because your app store rating will tank.”
Monitor Risk Zones
Roisi mentions that she has found the
Ethical Explorer Pack very useful. This is a pack of cards that articulates a number of risk zones where you might find the most risk of bias or harm. She has expanded her
“Black Mirror” test, where product managers must consider all the bad things that could happen if their product was used for the wrong purposes and which she has talked about at ProductTanks and on our podcast, into a workshop. She runs through the workshop in
this video.
Understand the Team Dynamic
Roisi also points out that it’s important that a product manager who comes into a business should remember that the people already there have been doing their best. “A mistake that I've made in the past is to be too heavy handed in pointing out potential ethical or bias issues.” It can make people feel threatened and uncomfortable, she says, because “as well as looking at your product and the way that your product can cause harm, you also have to introspect” and consider your own prejudices and biases. “By pushing other people to be more ethical in their practice, I'm also asking them to ask a lot of difficult questions about themselves,” says Roisi, “that can cause a reaction in some people, they can take it as a personal affront.”
You need to understand the team dynamic before you try to tackle any ethical questions, she says. “Learn how the team communicates so that you can speak their language, and then take a systematic approach tackling one element of the product at a time.” She suggests you start with what she calls the “ethical low-hanging fruit”: “Do you need to know someone’s gender or race? And if you do, how are you asking these questions? In a way that includes or excludes people? Find things that are tangible and easy for folks to understand.”
In addition, Roisi recommends that product people seek out the influencers as well as senior people when doing this work, “that might be a member of customer support that's been with the company a decade, and who everyone just listens to”.
This is such an important topic. If it feels overwhelming then perhaps the way to approach it is as a series of steps. Here’s a quick recap:-
- Think through your product’s unintended consequences by risk-mapping - map and sort your risks into buckets so you can prioritise them.
- Build in questions about ethics and accessibility throughout the product development process, don’t make them an afterthought. And build in time to think about the answers and challenges these questions present.
- Seek out hidden stakeholders, they may be just as affected by your product as your users.
- Consider the diversity on your team and the team dynamics - and do consult subject-matter experts.
- Don’t be heavy handed - you may be asking people to confront some difficult hard truths about themselves.
- Question, what’s the worst that could happen?
- As you’re thinking about making changes, go for quick and easy wins. They can make a huge difference to how people feel about your product, and demonstrate that your intentions are good.
Join the Discussion
We hope you'll help us to keep this conversation going and to share what you know too. Here are some easy ways to get involved:
Watch Our Product Ethics Panel
Mind the Product members can watch our panel,
How To Prioritise Ethical Practice as a Product Manager featuring Cennydd Bowles, Kathy Pham, Pavani Reddy, and Martin Eriksson in an in-depth discussion about Product ethics.
Watch now
Become a Mind the Product Member today and access the recording.
Share Your Case Studies
As Cennydd explains above, case studies will help to push forward the build responsibly message. If you have any experiences or case study material that would help to build a bank of resources on ethical product development, please get in touch. Email
editor@mindtheproduct.com.
Further Resources
