Does Conversation Hurt Or Help The Chatbot UX?

Chatbot fever has infected Silicon Valley. The leaders of virtually every tech giant — including Facebook, Google, Amazon and Apple — proclaim chatbots as the new websites, and messaging platforms as the new browsers. “You should message a business just the way you would message a friend,” declared Mark Zuckerberg when he launched the Facebook Messenger Platform for bots. He and the rest of the tech world are convinced that conversation is the future of business.

But is chatting actually good for bots? Early user reviews of chatbots suggest not. Gizmodo writer Darren Orf describes¹ Facebook’s chatbot user experiences as “frustrating and useless” and compares using them to “trying to talk politics with a toddler.” His criticisms are not unfair.

Here’s an example of a “conversation” I had with the 1–800-Flowers Messenger bot² after I became stuck in a nested menu and was unable to return to the main menu. Not exactly a pleasant or productive user experience.

³

To Chat Or Not To Chat? Link

Designers who are new to conversational interfaces often have the misconception that chatbots must “chat.” At the same time, they underestimate the extraordinary writing skill, technical investment and continual iteration required to implement an excellent conversational user experience (UX).

This article explores when conversation benefits and when conversation hurts the chatbot user experience. We’ll walk through case studies for both sides of the argument and compare divergent opinions from Ted Livingston, CEO of Kik, who advises⁶ bot makers to deprioritize open-ended chat, and Steve Worswick, the creator of “the most human chatbot,” who encourages developers to invest in truly conversational experiences.

As you’ll see from the examples below, both strategies can lead to successful chatbot experiences. The key is to choose the right level of conversational ability for your bot given your business goals, team capabilities and user needs.

The Case For Chat Link

Steve Worswick is the developer behind Mitsuku⁷, one of the world’s most popular chatbots. Mitsuku has twice won the Loebner Prize⁸, an artificial intelligence award given to the “most human-like chatbot.” The popular chatbot has conversed with more than 5 million users and processed over 150 million total interactions. 80% of Mitsuku’s users come back for more chats.

⁹

The longest a user has chatted with Mitsuku is nine hours in a single day — a testament to the bot’s extraordinary conversational abilities. Mitsuku does not help you find makeup products, buy flowers or perform any functional utility. The chatbot’s sole purpose is to provide entertainment and companionship. You won’t be surprised to find out that Worswick thinks “chatbots should be about the chat.”

Building a conversational chatbot that isn’t awful is extremely hard. Worswick nearly gave up many times when Mitsuku repeatedly gave unsatisfactory answers and users called her “stupid.” One major breakthrough occurred when Worswick programmed in a massive database with thousands of common objects such as “chair,” “tree” and “cinema,” along with their relationships and attributes.

Suddenly, Mitsuku could give sensible answers to strange user questions, such as, “Is a snail slower than a train?” or “Can you eat a tree?” According to Worswick, “Let’s say a user asks Mitsuku if a banana is larger than X, but she doesn’t recognize what X is. She knows that a banana is a relatively small object so can deduce that X is probably larger.”

Even if a chatbot is utilitarian, providing spontaneous answers in a conversation — especially if unexpected — can delight and engage users. Poncho¹² is a Messenger bot that gives you basic weather reports, but the creators gave the bot the personality of a Brooklyn cat. Poncho can conduct small talk and even recognizes other cats. “Weather is boring,” admits Poncho founder Kuan Huang. “We make it awesome.”

¹³

When You Should Add Conversation To Delight Users Link

Making a bot conversational takes tremendous effort, but if you are up to the challenge, here are the top situations in which conversation could distinguish your chatbot from competitors’ and truly delight users.

If You Need to Differentiate From Competition Link

As seen earlier, Poncho’s conversational personality distinguishes the chatty weather cat from boring, routine weather apps. Bots launch at a more rapid pace than mobile apps due to the lower technical barriers to entry. Dozens of bots already exist to service identical use cases, so winners need to stand out with a superior conversational UX.

Just like weather apps, public transit apps are soulless and boring. We use them out of necessity and not delight. Enter Bus Uncle¹⁶, a bot that can tell you anything you want to know about the Singaporean bus system in his quirky, broken English and suggest funny things to do while you wait.

¹⁷

Comprehensive, detailed guides and maps for the bus system exist on the Internet to help expats and locals find their way home, but Bus Uncle’s conversational interface both simplifies and adds joy to a routine task.

Beware that the bot is not all fun and games. Like any proper Asian uncle, Bus Uncle stays in character by occasionally forcing you to solve math problems.

²⁰

If You Need to Handle Edge Conditions Link

E-commerce is a challenging space for bots due to product diversity and language variability. Many conversational shopping bots malfunction when users use unrecognized vocabulary or suddenly switch contexts. Such failures are usually technical in nature, where a bot simply doesn’t have the requisite data set or intelligence to handle the edge input.

ShopBot²³ from eBay avoids common e-commerce bot UX failures by combining limited option menus with the ability to handle unexpected user input. While many shopping bots hem users into a narrow series of menus, ShopBot was able to quickly adapt when I switched from shopping for jeans to shopping for blouses.

²⁴

Shopping is a difficult use case for chatbots to master. Superior conversational experiences in e-commerce bots are a function not just of great copy, but of powerful technologies that process natural language, keep track of shoppers’ contexts and preferences, and anticipate diverse needs accurately.

RJ Pittman, chief product officer at eBay, explains²⁷, “Shoppers have complex needs, which are often not fully met by traditional search engines. The science of AI provides contextual understanding, predictive modeling, and machine learning abilities. Combining AI with eBay’s breadth of inventory and unique selection will enable us to create a radically better and more personal shopping experience.”

If You Can Humanize a Brand Link

Chatting is an intimate act we do with close friends and family, which is why chatting with a “brand” is often an awkward and strange experience. Strong conversational skills in a chatbot can overcome this barrier and establish an authentic connection.

Maintaining a consistent and compelling brand voice in chatbots is not easy. PullString²⁸, a conversational AI platform founded by ex-Pixar CTO Oren Jacob, employs an entire department of expert Hollywood screenwriters to bring brands like Mattel’s Barbie and Activision’s Call of Duty to life.

Its demo chatbot, Jessie Humani²⁹, is powered by over 3,500 lines of carefully selected dialog to create the impression that she’s your messed-up millennial friend who can’t get her life together without your help.

³⁰

The Case Against Chat Link

Many bot industry experts believe the word “chatbot” sets the wrong expectation among users that bots should have human-level conversational abilities. The hard reality is that natural-language processing and artificial intelligence still have much progress to make before bots will impress you with their gift of gab.

Ted Livingston, CEO of Kik, a popular messaging platform with a thriving bot store, is squarely on the side of no chatting. “The biggest misconception is that bots need to be about ‘chat.’ What we discovered is that bots that don’t have suggested responses simply don’t work. Users don’t know what to do with an empty input field and a blinking cursor,” he shared³³ at a recent bot conference.

Kik started building a conversational platform two years ago³⁴, long before bots suddenly became cool. In the beginning, its bots allowed freeform responses the same way Facebook Messenger bots do now. What resulted was user confusion and error, as well as complaints from developers about having to deal with the unnecessary complexity of processing open-ended conversation. Kik now restricts user responses to a limited set of predefined options and intentionally makes typing freeform text difficult.

For example, when Sephora’s Kik bot asks what type of beauty products a user would like to see, the bot follows the question with a menu of suggested responses to choose from. A user has to go out of their way to tap “Tap a message” in order to type normally.

³⁵

When You Should Restrict Chat For A Better UX Link

There are many cases in which designers of chatbots should restrict conversation to provide a superior experience. Below are a few common situations in which letting users type freeform conversational text complicates development and decreases your bot’s usability.

If User Error Would Lead to a Failed Transaction Link

1–800-Flower’s bot for Facebook Messenger originally gave users three options for flower delivery dates: “Today,” “Tomorrow” or “Choose another date.” The third option allowed users to type in dates freeform, which often resulted in error, confusion and an abandoned or failed transaction.

By removing the third option for users to type in a date manually, 1–800-Flowers actually increased the number of transactions and overall customer satisfaction. Restricting conversation helped it focus on its most important users, the ones who want to send flowers urgently.

³⁸

If Your Competitive Advantage Is Simplicity Link

Chatbots should give users the key advantage of completing tasks with fewer taps and context switches than regular mobile apps. Enabling open-ended chat can undermine this simplicity and add development complexity related to handling variable input.

An example is the simple meditation bot Peaceful Habit⁴¹ for Amazon Echo and Facebook Messenger. The bot is designed to help regular meditators build a daily practice and should be quicker to use than meditation apps.

On the Amazon Echo, a user can start a 5-, 10- or 20-minute meditation completely hands-free, with voice alone. On Facebook Messenger, the bot sends a daily reminder with limited user options, so only a single tap is required to start a meditation practice.

⁴²

If You Cannot Easily Handle Unbounded Input Link

Many user requests appear simple on the surface but are extremely complex to handle in an open-ended conversational interface due to variability of vocabulary, grammatical structures and cultural norms. For example, a user can ask to schedule a meeting by asking any of the following questions:

When’s Bob’s next open time slot?

Let me know the next three times Bob can chat.

Is Bob available at 4 PM PST today?

Turns out the complexity of handling seemingly simple meeting requests requires powerful artificial intelligence capabilities. Several well-funded companies have emerged just to solve narrow scheduling challenges with specialized technology.

When you consider more complex requests, such as asking for restaurant recommendations, limiting conversations often means less confusion for both your bot and your user. Sure⁴⁵, a bot that offers local restaurant recommendations, asks users to type in what they are craving, but it often can’t understand the responses.

⁴⁶

By contrast, a similar bot named OrderNow⁴⁹ finds local restaurants that deliver and offers a limited menu of cuisines to choose from.

⁵⁰

These examples demonstrate that complex artificial intelligence, machine learning or natural-language processing is not required to create a great user experience using a chatbot. As Ted Livingston, CEO of Kik, warns⁵³, “AI is not the killer app for bots. In fact, AI holds most bots back. Bots are just a better way to deliver a software experience. They should do one thing really well.”

How Much Should Your Chatbot Chat? Link

How “chatty” your chatbot should be will depend on your users’ mental models of chatbots and the goals and needs your chatbot fulfills for them. Bots on Kik that only offer limited responses can be just as successful and engaging as Mitsuku and Jessie Humani.

Problems occur when designers do not decide up front who their audience is, how the chatbot fits into their business or brand strategy, what domains the chatbot will and will not cover, and what a successful experience should look like.

When you are deciding how much “conversation” to design into your chatbot experience and are defining the right level of engagement, answer the following questions:

• How are you setting user expectations?
  
  If you brand your chatbot as a character or a human replacement, users will expect a minimum level of conversational ability. If your bot’s functionality is utilitarian or limited, then guide conversations towards specific outcomes.
• Is your chatbot utilitarian or entertainment-driven?
  
  Mitsuku is an artificial-intelligence companion, so she’s required to master the art of conversation. On the other hand, a Slackbot that performs SQL queries or pulls CRM data has no need to support chat.
• Does your chatbot reflect your brand’s voice?
  
  Major brands such as Disney⁵⁴ and Universal Studios⁵⁵ use chatbots to engage audiences beyond simple ad clicks and video views. A chatbot working as a brand ambassador needs to authentically reflect the domain and voice of the company it represents.
• Is your chatbot a familiar service or product?
  
  Businesses such as 1–800-Flowers and Domino’s Pizza already have millions of buyers who use their websites, mobile apps and phone numbers to order products. Users who already know what you offer and what they like won’t require as much explanation and hand-holding.
• Does your chatbot need to differentiate itself in a competitive market?
  
  Weather apps are a dime a dozen. Poncho the Weather Cat differentiates itself by having a distinct personality and delightful reactions, making the bot stand out against other weather services.
• How strong is your technical team and AI platform?
  
  Building an adaptable and user-friendly conversational AI is incredibly challenging. Worswick invested over a decade to make Mitsuku the award-winning chatbot she is today. Each conversational AI platform has strengths and weaknesses that will affect your chatbot’s UX.
• How strong is your writing team?
  
  In the world of bots, writers are the new designers. Do your writers understand how to write engaging, emotional copy that draws users in? Bots reflect the communication skills of their makers.

As natural-language understanding, machine learning and artificial intelligence improve, chatbots will inevitably become smarter and more capable in interactions with humans.

For now, just be sure that your bot either sticks with utilitarian offerings or stays within a comfortable zone of conversational topics. Take a cue from how Mitsuku gracefully avoids confrontation by excusing herself from a potentially awkward political conversation.

⁵⁶

(al)

Footnotes Link

1. 1 http://gizmodo.com/facebook-messenger-chatbots-are-more-frustrating-than-h-1770732045
2. 2 https://www.messenger.com/t/1800flowers/
3. 3 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_1_flowers_brokenrecord_cut-opt.png
4. 4 http://www.topbots.com
5. 5 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_1_flowers_brokenrecord_cut-opt.png
6. 6 http://www.topbots.com/bot-discovery-virality-lessons-kik-ceo-ted-livingston/
7. 7 http://www.mitsuku.com
8. 8 https://en.wikipedia.org/wiki/Loebner_Prize
9. 9 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_2_mitsuku_goodconversationalist-opt.png
10. 10 http://www.topbots.com
11. 11 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_2_mitsuku_goodconversationalist-opt.png
12. 12 https://www.messenger.com/t/hiponcho
13. 13 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_3_poncho_othercats-opt.png
14. 14 http://www.topbots.com
15. 15 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_3_poncho_othercats-opt.png
16. 16 https://www.facebook.com/sgbusuncle/
17. 17 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_4_busuncle_missedbus-opt.png
18. 18 http://www.topbots.com
19. 19 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_4_busuncle_missedbus-opt.png
20. 20 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_5_busuncle_forcedmath-opt.png
21. 21 http://www.topbots.com
22. 22 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_5_busuncle_forcedmath-opt.png
23. 23 https://shopbot.ebay.com/
24. 24 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_6_ebay_ShopBot-opt.png
25. 25 http://www.topbots.com
26. 26 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_6_ebay_ShopBot-opt.png
27. 27 https://www.ebayinc.com/stories/news/say-hello-to-ebay-shopbot-beta/
28. 28 https://www.pullstring.com/
29. 29 https://www.messenger.com/t/jessiehumani
30. 30 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_7_jessie_pullstring_mess-opt.png
31. 31 http://www.topbots.com
32. 32 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_7_jessie_pullstring_mess-opt.png
33. 33 http://www.topbots.com/bot-discovery-virality-lessons-kik-ceo-ted-livingston/
34. 34 https://backchannel.com/how-kik-predicted-the-rise-of-chat-bots-2eaf9027b86e#.qwhkcf3ut
35. 35 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_8_kik_sephora_suggestedresponses-opt.png
36. 36 http://www.topbots.com
37. 37 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_8_kik_sephora_suggestedresponses-opt.png
38. 38 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_9_1800flowers_restricted_options-opt.png
39. 39 http://www.topbots.com
40. 40 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_9_1800flowers_restricted_options-opt.png
41. 41 http://www.topbots.com/peacefulhabit
42. 42 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_10_peaceful_habit_limited_options-opt.jpg
43. 43 http://www.topbots.com
44. 44 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_10_peaceful_habit_limited_options-opt.jpg
45. 45 https://www.messenger.com/t/surebot/
46. 46 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_11_sure_confused_bot-opt.png
47. 47 http://www.topbots.com
48. 48 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_11_sure_confused_bot-opt.png
49. 49 https://www.messenger.com/t/ordernowbot
50. 50 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_12_ordernow_limitedoptions-opt.png
51. 51 http://www.topbots.com
52. 52 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_12_ordernow_limitedoptions-opt.png
53. 53 http://www.topbots.com/bot-discovery-virality-lessons-kik-ceo-ted-livingston/
54. 54 http://www.topbots.com/project/disney-zootopia/
55. 55 http://www.topbots.com/project/universal-unfriended/
56. 56 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_13_mitsuku_donaldtrump-opt.png
57. 57 http://www.topbots.com
58. 58 https://www.smashingmagazine.com/wp-content/uploads/2016/11/topbots_chatbots_13_mitsuku_donaldtrump-opt.png

↑ Back to topTweet itShare on Facebook