Tricks and Traps When Building Conversational Products

BY Rajat Harlalka ON DECEMBER 18, 2018

For a long time, we’ve thought of interfaces strictly in a visual sense: buttons, drop-down lists, sliders, carousels (please no more carousels). But now we are staring into a future composed not just of visual interfaces, but conversational ones too.  In the first part of this series, I looked at some tips and best practice to design conversational flows for such interfaces. Now I’d like to look at other aspects to consider when building conversational products.

It’s all About Context

Once you’ve generated potential use cases and filtered them against real business goals, you need to answer a few questions about your target users. These include the customer’s identity, feedback, the channel used for contact, date, time, location, and other information in real time.

All these questions will influence the implementation of a conversational interface experience. By analyzing all that data using artificial intelligence and machine learning, the bot can anticipate customer needs. In 2D design contexts, you can watch a user interact with your app — scrolling, tapping buttons, typing – but in conversational product design, you have to look at the broader user context.

Intent is easier to express through words than a hamburger menu, but natural language is harder to interpret than a click. It gets easier over time,through the  understanding of context and observing user behavior.

It is one thing to train humans to input commands that a computer understands, but a completely different process to teach a computer to map natural human language inputs to intent. Mapping natural language to intent is the first layer to conversational UI.

Leverage Your Platform’s Capabilities

When one person asks another one a question, it takes an average of 200 milliseconds for them to respond. This is so fast that we can’t even hear the pause – in fact, it’s faster than our brains work. It takes the brain about half a second to retrieve the words to say something, which means that in conversation, one person is gearing up to speak before the other is even finished. By listening to the tone, grammar, and content of another’s speech, we can predict when they’ll have finished.

A conversational product should try to get the most out of the platform it is on, however, bots vary completely from platform to platform. All the platforms have different functionality, with different feature sets or capabilities.

Alexa is a voice-first experience. Alexa’s versions of apps are called “skills” – the idea being that you can teach your Alexa to do something new by enabling a new “skill”, created by a third party.

Google’s Assistant is more of a platform: “Actions” (Google’s version of “skills”) can be either voice-activated, tapped on or typed to. A hybrid voice, text, and type interface, Assistant lets you use whatever method suits you in your current context and move between methods seamlessly. This also means Assistant is available on a wider range of form factors, from smart speakers to watches to smartphones to TVs.

Facebook Messenger is a chat platform, augmented with a wide selection of customisable widgets allowing users to shortcut conversations and quickly get stuff done. It’s really easy to bring a Messenger chatbot into a human conversation to help you get stuff done.

Monitor the Health of the bot

The older analytics tools don’t cut it when it comes to bots, so it’s important not just plug in something off-the-shelf. There are new players, for example Botmetrics and Dashbot, who are building the tools for bots from first principles, and they are improving every day. But even they may not provide all the information and functionality you need, and you should be careful that you don’t end up with your data being silo-ed and inaccessible. Below are some of the elements and potential pitfalls in a conversational product that you should be aware of:

  • Confusion Triggers: The entire funnel is fashioned with ways for things to go wrong. Given the range of possible user inputs, bots often misinterpret or simply can’t determine what the user wants. Examine the previous prompts to see where you could make some clarifications. Was the call to action clear?
  • Conversation Steps: A conversation step is a single back-and-forth exchange between user and bot. Knowing the average conversation steps enables you to start segmenting good or bad experiences between the user and the bot. Your average steps will be different depending on the type of the bot. Conversations that either significantly exceed or fall short of the average conversation step usually indicate a bad user experience. Either a user gave up too quickly or a bot took too long to complete a user’s goal.
  • Conversation Flow:
    • Pay attention to the way users naturally ask for things. Does the flow make users feel like they can only provide one piece of information at a time, or does it encourage them to provide multiple details in one sentence?
    • Look for places where users look confused or are unsure of what to say or do.
    • Users might say something you didn’t expect. Take note of it and add the handling for it in your design.
    • Signs of frustration or impatience show that the interaction is too long-winded. Review your prompts to see if you can be more concise.
    • Observe who’s  speaking the most and whether users seem to be in control of the conversation
  • Conversations per User: Associated with the number of average steps is the average number of conversations per user. This indicates engagement with the bot.
  • Active vs Engaged Users: If your bot supports both push and pull interactions, then you will want to compare the differences in how users interact with your bot. Active users will consume messages sent by the bot, whereas engaged users will respond. This is important because you can learn from users who are engaged. These are your happy users who are leaning into the experience. You will want to accommodate these types of conversations.
  • Response Time: Know how long it takes for your bot to respond. Know the latency period between command and response. This affects the overall user experience. You’re looking for median time as well as outliers here.
  • A bot should save time and relieve stress by reducing friction and effort, otherwise it is not much better than the website or app that came before it. Integrations and contextual relevance are key elements to the experience.

    A Hybrid Experience is OK

    It’s a mistake to believe a conversational design means that every user input and output has to be handled in a purely chat format. Chatbots live within messaging apps and users are already accustomed to communicating with friends using text, but text input is not efficient for all use cases. The GUI replaced the text-based terminal for a reason, and some tasks like browsing and selecting are faster by touch or click. Whenever it makes sense, it’s fine to consider a hybrid approach and use other interactive elements within the conversation (e.g. buttons, cards, forms) to help users to complete their goals faster and with less friction.

    Over-reliance on structured messages, however, will feel artificial as you lose the conversational element.

    Conclusion

    Designing from first principles is the only way to avoid fundamental usability problems with a conversational interface, especially because it’s a new paradigm for user experience. Reasonably safe assumptions in web or mobile apps, where people already know the standard UIs and gestures, don’t apply in the conversational interface world.

    Conversational products relate to delivering service over voice and messaging interfaces, but they also imply a different philosophy, which is much more about understanding the context, constant iteration, and constant A/B testing, when it comes to product development.

About

Rajat Harlalka

Rajat is a product manager and technology evangelist with close to 12 years of experience in the computer software industry in technology, marketing and strategy roles. Follow him on Twitter as @RajatHarlalka.