When Google first demonstrated its AI phone-calling technology Duplex back in May, the pre-recorded demo struck many observers as eerie. Piped through the speakers on stage at the Google I/O developer conference while a video capture of an Android phone played on screen, we heard an artificial voice call both a hair salon and a restaurant to book reservations on behalf of a human.
Right away, many in the tech community cited two big problems. First, the people on the receiving end of the call were unaware that the voice speaking into the phone was a machine, meaning Duplex was essentially fooling unsuspecting humans. Second, the bot in the demo never indicated it was recording the phone call, raising the eyebrows of privacy advocates and prompting follow-up questions from journalists (including writers at WIRED).
On Tuesday, Google gave multiple demonstrations of its Duplex technology in action. This time, there were some obvious differences.
Now, just a couple weeks ahead of Duplex’s rollout among a small set of users and businesses, Google is trying to give its phone-calling robot a do-over. The company is attempting to prove it has addressed some of the concerns about Duplex. And its latest pitch around transparency is coming at a time when some of its more critical use cases for AI are being seriously questioned—just recently, the company released a set of AI principles prohibiting Googlers from using AI in technologies that could violate human rights or cause “overall harm.”
On Tuesday, at a hummus shop in Mountain View, California just down the road from Google’s headquarters, the company gave multiple demonstrations of its Duplex technology in action. This time, there were some obvious differences. “Hi, I’m calling to make a reservation,” the bot said, which Google patched through speakers in the shop so the assembled reporters could hear it. “I’m Google’s automated booking service, so I’ll record the call. Can I book a table for Thursday?”
Google executives Nick Fox and Scott Huffman, along with product manager Valerie Nygaard, were on-hand to answer questions from reporters. Nygaard even had reporters rotate through the host’s stand at the front of the shop and take turns answering the phone, so we could interact with the Duplex-powered virtual assistant calling the restaurant. Each of the Duplex calls were being initiated by a Google Assistant request off of a laptop, just feet away in the restaurant.
I’ll admit that when I answered the phone at Oren’s Hummus Shop, I tried hard to trip up the Duplex bot. A female-sounding voice called and asked for a reservation Monday the 2nd. After determining that “she” meant the 2nd of July, I asked for the number of people in her party and for the desired time. “At 9pm,” she replied. I told the bot that the shop closes at 9:30 pm—making it up as we went along—so she might want to book for an earlier time. 7:30pm, the bot suggested? “We have something at 7:45, actually,” I said.
I then asked whether there were any allergies in the group. “OK, so, 7:30,” the bot said. “No, I can fit you in at 7:45,” I said. The bot was confused. “7:30,” it said again. I also asked whether they would need a high chair for any small children. Another voice eventually interjected, and completed the reservation.
I hung up the phone feeling somewhat triumphant; my stint in college as a host at a brew house had paid off, and I had asked a series of questions that a bot, even a good one, couldn’t answer. It was a win for humans. “In that case, the operator that completed the call—that wasn’t a human, right?” I asked Nygaard. No, she said. That was a human who took over the call. I was stunned; in the end, I was still a human who couldn’t differentiate between a voice powered by silicon and one born of flesh and blood.
Talking Back
I asked Huffman and Fox whether Google regretted showing off a carefully-produced Duplex demo back in May that offered little in terms of transparency or exposition. Fox didn’t say directly whether he regretted it. “We thought of the demo at I/O as much more of a technology demo, whereas what you see here is much more of the product side of the technology,” Fox said. “It was more of a pure technology demo. But we always knew we needed disclosure and it was the right thing to do.” Fox added that Google found all of the feedback from people “useful.”
While Google has addressed the stickiest stuff in that demo—adding a statement identifying the caller as a bot and disclosing the recording of the phone call—one big unanswered question about Duplex is one of agency: Who is responsible when a bot calls a business but then a human flakes?
“The agency question to me is the most complex, and will probably take the longest for us to work out as a society,” says Roman Kalantari, senior design director of creative technology at the design consultancy Fjord. “Will people feel less pressure to show up to an appointment their bot made because they never spoke to a person? This is already a huge problem at restaurants, for example, and this will likely get worse when it is easier, and the user has even less emotional attachment to the interaction or guilt about cancelling or not showing up.”
Bot Beginnings
During Tuesday’s demo, Huffman gave the group some background on the development of Duplex—its earliest phone calls, the human operators who back it up, and why Google sees Duplex’s tech evolving with use the same way self-driving car systems do. Huffman said it only took “a couple months” for the initial version of Duplex to get set up, but its earliest demos were incredibly rudimentary, with the speaker of a wired telephone being placed next to a Mac laptop’s speakers while the Duplex technology ran on the machine.
Huffman played one of the first Duplex phone calls ever made, when the bot tried to reserve a table at a restaurant. It was awkward. There was some confusion when the human being on the phone asked about the reservation time, and again when the human asked for the first name of the reserving party. The Duplex-powered bot was clearly flustered. “It wasn’t super good,” Huffman admitted, “but we could tell it had potential.”
Google began to employ human moderators who would annotate the earliest Duplex calls. This team would take those notes and feed them into the system, allowing the AI to learn and adjust. Those human moderators are still working on Duplex—in fact, some of them are operators who will save a Duplex call when things go sideways—but Huffman and Fox declined to say how many people they’ve hired for the Duplex team. Google has also been studying speech disfluencies, and how they relate to Duplex, Huffman said. How should a bot deal with uncertainty in a polite way? How frequently should it offer conversational acknowledgement—the “Mmhmm”s we all say when someone’s been rambling for awhile—over the phone?
One way Google is trying to position Duplex is in the same realm as a self-driving car—an analogy that might be more welcome right now than an association with Google’s controversial military AI program. There’s a manual mode, in which the human’s hands grip the wheel, or, in this case, when a human makes the phone call. Then there’s a supervised mode, and then, “maybe the system is good enough where you can sit back and let the car drive itself,” Huffman said. “Four out of five of the calls we work on can be automated completely.”
Your Call
Google still hasn’t said when it will officially roll out Duplex to a wide user base, just that public tests of it are going to start in the next couple of weeks, with a “limited set of trusted testers and select businesses.” It also won’t say how many testers or businesses there are, to start. Duplex will work as part of Google Assistant, the company’s virtual assistant for phones and smart speakers. Initially, it will respond to requests around holiday hours for businesses; over the next few months, it will expand to include restaurant reservations and hair salon appointments.
Much of Google’s focus during Tuesday’s Duplex demo was around how it could help businesses. According to Google’s own internal research, 60 percent of small businesses that take reservations don’t have an online booking system. Huffman says telling people to pick up the phone and call some place is a barrier in an age when so many tasks like booking appointments and placing orders can be done online. Google thinks it can fix this resistance to making phone calls and help those businesses that still do things the old fashioned way.
Huffman said there was an interpretation after the demo at Google I/O back in May that Google’s AI could be used to take over any conversation. “This is trained for specific tasks,” he said. “I really want to make clear that the reason why it works is that we’ve chosen very specific tasks … it’s not a general purpose AI, but it’s very good at doing these narrow and specific things.”
Huffman makes a good point, but it may not be the “specific things” that continue to raise questions about Duplex. Rather, it may be the “very good” part.
More Great WIRED Stories
- The reason you use closed captions for everything now
- The stock market loophole that screws the little guy
- Inside Palmer Luckey’s bid to build a border wall
- The bike share war is shaking up Seattle like nowhere else
- Here are the best Mac alternatives for Windows users
- Looking for more? Sign up for our daily newsletter and never miss our latest and greatest storieso