Automated ‘voices’ that were supposed to do mundane tasks online also now spread hate speech and polarise opinion. Are they a boon or a threat?
Alan Turing’s famous test of whether machines could fool us into believing they were human – “the imitation game” – has become a mundane, daily question for all of us. We are surrounded by machine voices, and think nothing of conversing with them – though each time I hear my car tell me where to turn left I am reminded of my grandmother, who having installed a telephone late in life used to routinely say goodnight to the speaking clock.
We find ourselves locked into interminable text chats with breezy automated bank tellers and offer our mother’s maiden name to a variety of robotic speakers that sound plausibly alive. I’ve resisted the domestic spies of Apple and Amazon, but one or two friends jokingly describe the rapport they and their kids have built up with Amazon’s Alexa or Google’s Home Hub – and they are right about that: the more you tell your virtual valet, the more you disclose of wants and desires, the more speedily it can learn and commit to memory those last few fragments of your inner life you had kept to yourself.
Read more
As the line between human and digital voices blurs, our suspicions are raised: who exactly are we talking to? No online conversation or message-board spat is complete without its doubters: “Are you a bot?” Or, the contemporary door-slam: “Bot: blocked!” Those doubts will only increase. The ability of bots – a term which can describe any automated process present in a computer network – to mimic human online behaviour and language has developed sharply in the past three years. For the moment, most of us remain serenely confident that we can tell the difference between a human presence and the voices of the encoded “foot soldiers” of the internet that perform more than 50% of its tasks and contribute about 20% of all social media “conversation”. That confidence does not extend, however, to those who have devoted the last decade or so to trying to detect, and defend against, that bot invasion.
Naturally, because of the scale of the task, they must enlist bots to help them find bots. The most accessible automated Turing test is the creation of Professor Emilio Ferrara, principal investigator in machine intelligence and data science at the University of Southern California. In its infancy the bot-detector “BotOrNot?” allowed you to use many of the conventional indicators of automation – abnormal account activity, repetition, generic profiles – to determine the origin of a Twitter feed. Now called the Botometer(after the original was targeted by copycat hacks), it boasts a sophisticated algorithm based on all it has learned. It’s a neat trick. You can feed it your own – or anyone else’s – Twitter name and quickly establish how bot-like your bon mots are. On a scale where zero is human and five is machine, mine scored 0.2, putting @TimAdamsWrites on a sentient level with @JeremyCorbyn, but – disturbingly – slightly more robotic than @theresa_may.
Chatbots for health, wealth and music
WoeBot (pictured)
Designed to help those suffering from depression by facilitating quick conversations. It will even check up on you every now and then to see how you are doing. The company bills it as a “robot friend, who’s ready to listen”.Cleo
An AI chatbot aimed at helping you to organise your finances. It connects with your bank account and can give you detailed information via Facebook Messenger about what you spent and where you spent it.
Robot Pires
Arsenal FC invite you to talk to a cartoon-bot Roberto Pires – ask for news about the club, as well as the player’s own record, including how many goals he scored for the Gunners.
Paul McCartney
The “official Messenger bot for the music legend Paul McCartney” will react with gifs of the singer and can tell you when he’s on tour, about his latest projects, and more. However, it doesn’t respond too well to questions. When asked “How old is Paul?” the bot replied with a video of a flying baguette.
TfL TravelBot
Designed to allow you to check on how London’s transit system is running, all via Facebook Messenger. It can be asked about the status of lines, and when asked how to get from A to B will provide three links with the fastest route.
Lark
A health coach that can help users manage the symptoms of hypertension, diabetes etc. Using data gathered from the user’s connected devices, it makes data-driven nudges and recommendations to encourage healthier behaviour.
OllyBot
The Olly Murs official chatbot can answer questions about the singer, provide fans with information about his upcoming tours and offer playlists of his music. The bot replicates the celebrity’s tone by calling himself “29+3” years old and ending messages with winking emojis.
Insomnobot-3000
Created by mattress company Casper, this chatbot allows sleepless users to message it for recommendations and suggestions that may improve their sleeping routine.
Harry Lye
Photograph: WoebotWas this helpful?Advertisement
Speaking to me on the phone last week, Ferrara explained how in the five years since BotOrNot has been up and running, detection has become vastly more complex. “The advance in artificial intelligence and natural language processing makes the bots better each day,” he says. The incalculable data sets that Google and others have harvested from our incessant online chatter are helping to make bots sound much more like us.
The Botometer is powered by two systems. One is a “white box” that has been trained over the years to examine statistical patterns in the language, Ferrara says, “as well as the sentiment, the opinion,” of tweets. In all there are more than 1,200 weighted features that a Twitter feed is measured against to determine if it has a pulse. Alongside that, the Botometer has a “black-box model” fed with a mass of data from bots and humans, which has developed its own sets of criteria to separate man from machine. Ferrara and his team are not exactly sure what this system relies on for its judgments, but they are impressed by its accuracy.
When Ferrara started on this work, he felt he had developed his own sixth sense for sniffing out artificial intelligence on Twitter. Now he is no longer so confident. “Today it is not clear to me that I interact with as many humans as I thought I did,” he says. “We look hard at some accounts, we run them through the algorithm and it is undecided. Quite often now it is a coin toss. The language seems too good to be true.”Advertisement
Not all bots aim to deceive; many perform routine operations. Bots were originally created to help automate repetitive tasks, saving companies money and time. Some bots help to refresh your Facebook feed, or keep you up to date with the weather. On social media, bots were originally coded to search for hashtags and keywords and retweet or amplify messages: “OMG have you seen this?!” They acted as cheerleaders for Justin Bieber or Star Wars or Taylor Swift. There were “vanity bots” which added numbers and fake “likes” to profiles to artificially enhance their status, and “traffic bots” designed to drive customers to a particular shopping site. There were also bots that acted as grammarians, making pedantic corrections to tweets, or simple gags like Robot J McCarthy, which sought out conversations using the word “communist” and replied with a nonsensical slogan.
At some point political bots entered the fray, mostly on Twitter, with the intent of spreading propaganda and misinformation. Originally these seem to have been the work of individual hackers, before the techniques were adopted by organised and lavishly funded groups. These bots, Ferrara suggests, proved to be a highly efffective way to broadcast extremist viewpoints and spread conspiracy theories, but also were programmed to search out such views from other, genuine accounts by liking, sharing, retweeting and following, in order to give them disproportionate prominence. It is these bots that the social media platforms have been trying to cull in the wake of investigations into the 2016 American election by Robert Mueller and others. Twitter has taken down a reported 6m bot accounts this year.
Advertisement
When I spoke to Ferrara he was looking at the data from the American midterm elections, examining the viral spread of fake news and the ways in which it was still being “weaponised” by battalions of automated users. “If you were an optimist you would think that the numbers look OK,” he says. “Between 10 and 11% of the users involved in conversations around the election are flagged as bots – and that is significantly less than in 2016 when it was something like 20%. The pessimistic interpretation is that our bot-detection systems are not picking up the more sophisticated bots, which look just like humans even to the eyes of the algorithms.”
The unseen global army of “bot herders”, those shadowy individuals and corporations and rogue government agencies that send their bots out into the virtual world, have a couple of advantages in this latter respect. One is that they are now able to find enormous amounts of natural-language data to develop the next generation of talkative bots. The other is that these creations can exploit our tendency to ascribe trusted human characteristics to voices even if, on a rational level, we suspect that they are artificial. That psychology is as old as electronic communication itself.
All modern chatbots trace their family tree back to the experiments by Joseph Weizenbaum with Eliza, named after Ms Dolittle in Pygmalion for “her” ability to master received pronunciation. In 1966, Weizenbaum, a German-American professor at the Massachusetts Institute of Technology, created a prototype chatbot that searched for keywords in conversations conducted with humans typing at keyboards. The rudimentary program would pick up these words and use them in its reply. If it did not locate a useful word, it would offer a neutral response. Weizenbaum set up Eliza to mimic the tone and manner of an east coast psychotherapist, a dynamic that allowed its voice to “adopt the pose of knowing almost nothing of the real world”.
The professor had assumed that his bit of trickery would be a useful way of teaching his students about the structures of language and dialogue. What he discovered however, was that several of them became hooked on conversing with Eliza and telling her their troubles, opening up to the program as if it were a therapist. When Weizenbaum’s secretary asked him to leave the room one morning because she was in the middle of a session with Eliza, he became alarmed by the general willingness to be so seduced by a machine voice. He published an academic paper which included the first notable conversation between woman and machine. It went like this.
Woman: Men are all alike.
Eliza: In what way?
W: They’re always bugging us about something or other.
E: Can you think of a specific example?
W: Well, my boyfriend made me come here.
E: Your boyfriend made you come here?
W: He says I’m depressed much of the time.
E: I am sorry to hear that you are depressed.
W: It’s true. I am unhappy.
E: Do you think coming here will help you not to be unhappy?
W: I need some help, that much seems certain.
E: What would it mean to you if you got some help?
W: Perhaps I could learn to get along with my mother.
E: Tell me more about your family.
W: My mother takes care of me.
E: Who else in your family takes care of you?
W: My father.
E: Your father?
W: You are like my father in some ways.
E: What resemblance do you see?
W: You are not very aggressive, but I think you don’t want me to notice that.
E: What makes you think I am not very aggressive?
W: You don’t argue with me…Advertisement
Weizenbaum later became a fierce critic of the ways that human intelligence was becoming in thrall to artificial intelligence, which he viewed as a corporatised, reactionary force. He was stunned by how quickly the users of Eliza gave human agency to what was a relatively simple piece of code. It indicated to him that the brain had evolved to view all speech as meaningful, even if it came from a patently fake source. He worried, extremely presciently, about the implications of this: “The whole issue of the credibility [to humans] of machine output demands investigation,” he concluded in his paper. “Important decisions increasingly tend to be made in response to computer output. Eliza shows, if nothing else, how easy it is to create and maintain the illusion of understanding.”
The many progeny of Eliza have evolved into chatbots – bits of software designed to mimic human conversation. They include recent entries into the annual Loebner prize, which offers chatbot contestants the chance to fool a panel of human judges with their intelligence. The comforting principle of telling our deepest fears to a machine is also exploited in various “therapy” platforms, marketed as a genuine alternative to conventional talking cures. Each of them trades on the idea of our fundamental desire to be listened to, the impulse which shapes social media.
Lisa-Maria-Neudert is part of the computational propaganda project at Oxford University, which studies the ways in which political bots have been used to spread disinformation and distort online discourse. She argues that the seductive intimacy of chatbots will prove to be the next battleground in this ongoing war.Advertisement
The Oxford research team began examining the huge growth of bot activity on social media after the shooting down of the MH17 passenger plane with a Russian missile in 2014. A dizzying number of competing conspiracy theories were “seeded” and encouraged to spread by a red army of automated agents, muddying the facts of the atrocity. The more Oxford researchers looked, the more they saw how similar patterns of online activity were amplifying specific hashtags or distorting news.
The most striking thing to me to this day is that people are really, really bad at assessing the source of informationEmilio Ferrara
In the beginning, Neudert suggests, the bots would rely on volume. “For example,” she says, “in the Arab spring bots were flooding hashtags that activists were using underground in order to make the conversation useless.” Or, like Eliza, bots would respond to a keyword to get a marginal topic trending, and, often, into the news. This was an effective but blunt instrument. “If I tweet something saying ‘I hate Trump’,” Neudert explains, an old-style bot “would send me a message about Trump because it is responding to that keyword. But if I say ‘I love Trump’, it would send me the same message.” These bots were not smart enough to recognise intent, but that is changing. “The commercial companies that are using artificial intelligence and natural language processing right now are already building such technologies. What we are doing as a project is to try to find out if the political actors are already using them also.”
Neudert is particularly interested in the new generation of branded chatbots that push content and initiate conversations on messaging platforms. Such chatbots – which openly declare themselves to be automated – represent a new way for businesses and news services to attract your attention, giving the impression of speaking just to you. She imagines the propaganda bots will use the same technology, but without declaring themselves. “They’ll present themselves as human users participating in online conversation in comment sections, group chats, and message boards.”
Advertisement
At present the feasibility of a truly conversational chatbot, one that can understand the context of any conversational gambit, pick up tonal ambiguities and retain a sense of how the discussion is evolving, is still a long way off. The new generation of chatbots might be good at answering direct questions or interrupting debates, but they are ill-equipped to sustain coherence over a range of subjects.
What they may soon be capable of is maintaining short bursts of plausible dialogue with a predetermined narrative. In a recent paper in the MIT Review, Neudert suggests that in the near future such “conversational bots might seek out susceptible users and approach them over private chat channels. They’ll eloquently navigate conversations and analyse a user’s data to deliver customised propaganda.” In this scenario, and judging by what is already happening, the bots will have the capacity to “point people towards extremist viewpoints, counter arguments in a conversational manner [and] attack individuals with scripted hate speech, overwhelm them with spam, or get their accounts shut down by reporting their content as abusive.” And of course all of this will be done by a voice that engages one on one, that talks just to us.
There are a number of fast-growing companies that are beginning offer the kind of technology that Neudert describes, as a legitimate marketing tool. Several are official partners of Facebook in order to use its Messenger service. They include the market-leading Russian-based company Chatfuel, which has enabled thousands of organisations to build Messenger chatbots, including headline acts such as the NFL and the Labour party, and a number of smaller operations such as Berlin-based Spectrm, which has created Messenger chatbots for the likes of CNN and Red Bull.
I spoke to Max Koziolek, one of the founders of Spectrm, who is (predictably) evangelical about this new way of businesses speaking “like a friend” to their users and customers.
Using a combination of natural language data and human input, Spectrm has created bots that can already converse on a narrow range of subject matter. “On a specialist subject you can now get to 85% of queries pretty fast,” Koziolek says, “and then you will have the long tail, all those surprise questions which take ages to get right. But if you are making something to answer queries about Red Bull, for example, does it really need to know who is the chancellor of Germany?”
One of the most successful chatbots Spectrm has created was a public health initiative to advise on the morning-after contraceptive pill. “It is one of those times when someone might prefer to speak to a bot than a human because they are a bit embarrassed,” Koziolek says. “They talk to the bot about what they should do about having had unprotected sex and it understands naturally 75% of queries, even if they are writing in a language which is not very clear.” Within a year of listening and learning, he is confident that capacity will have increased to nearly 100%.
Increasingly we will become used to almost every entity in our lives “talking to us as if it is a friend”, he suggests, a relationship that will require certain rules of engagement. “If you send messages after 11pm that’s bad. And also if you send too many. I wouldn’t send more than two messages a day as a publisher, for example. It’s a very intimate space. A friend is sending me relevant information and at the right time.”
Far from being a new frontier in the propaganda wars, Koziolek believes – hugely optimistically – that such direct conversation could help to clear the internet of hate speech, giving users more control over who they hear from.
Does it matter whether they know that the chat is from a machine?
“We don’t see big differences,” he says. “Sometimes our bots have a clear personality and sometimes they don’t. Bots which have a personality will always say goodnight, for example. Or ‘How are you?’”
Do those types of bots produce longer conversations?
“Different kinds of conversations. Even though you know this thing is a robot, you behave differently toward it. I would say you cannot avoid that. Even though you know it is a machine, you immediately talk to it just like it is a human.”
This blurring of those lines is less welcome to observers like Ferrara, who has had a front-row seat in the changing dialogues between human and machine. I wonder, in observing at close quarters for so long if he has, anecdotally, felt the mood of conversations changing, if interactions have become angrier?
He says he has. “The thing was, I was becoming increasingly concerned about all sorts of phenomena,” he says. “I worked on a variety of problems, bots was one. I also looked at radicalisation, at how Twitter was being used to recruit Isis and at how conspiracies affected people’s decision-making when it comes to public health, when it comes to vaccines and smoking. I looked at how bots and other campaigns [that] had been used to try to manipulate the stock market. There are all sorts of things that have nefarious consequences.”
What aspect of this behaviour alarmed him the most?
“The most striking thing to me to this day is that people are really, really bad at assessing the source of information,” he says. One thing his team have shown is that the rate at which people retweet information from bots is identical to that from humans. “That is concerning for all sorts of reasons.”
This is not a problem you can solve with technology alone. You need regulationEmilio Ferrara
Despite the revelation of such findings, he gets frustrated that people, for political purposes, still seek to dismiss the ways in which these phenomena have changed the nature of online discourse. As if the most targeted propaganda, employed on the most unregulated of mass media, had no effect on opinion or behaviour.
One of his later projects has been to try to show how quickly messages can spread from, and be adopted by, targeted user groups. Last year, Ferrara’s team received permission to introduce a series of “good” health messages to Twitter via bots posing as humans. They quickly built up thousands of followers, revealing the ways in which a flood of messages, from apparently like-minded agents, can very quickly and effectively change the tone and attitude of online conversation.
Unfortunately, such “good” bots are vastly outnumbered by those seeking to spread discord and disinformation. Where does he place his faith in a solution?
“This is not a problem you can solve with technology alone,” he says. “You need tech, you need some system of regulation that incentivises companies to do that. It requires a lot of money. And then you need public opinion to care enough to want to do something about it.”
I suggest to him that there seems to be a grain of hope in the fact that people are reaching out in greater numbers toward trusted, fact-checked news sources: subscriptions to the New York Times and the Washington Post are up (and the Guardian and Observer just notched up a million online supporters).
“It’s true,” he says. “But then I have a chart on my screen which I am looking at as I talk to you. It gives live information on the sources of things being retweeted by different groups. Way at the top is Breitbart: 31%. Second: Fox News. Then the Gateway Pundit [a far-right news site]. Looking at this,” he says, “it is like we haven’t yet learned anything from 2016.”
Since you’re here…
… we have a small favour to ask. More people are reading and supporting our independent, investigative reporting than ever before. And unlike many news organisations, we have chosen an approach that allows us to keep our journalism accessible to all, regardless of where they live or what they can afford.
The Guardian is editorially independent, meaning we set our own agenda. Our journalism is free from commercial bias and not influenced by billionaire owners, politicians or shareholders. No one edits our editor. No one steers our opinion. This is important as it enables us to give a voice to those less heard, challenge the powerful and hold them to account. It’s what makes us different to so many others in the media, at a time when factual, honest reporting is critical.
Every contribution we receive from readers like you, big or small, goes directly into funding our journalism. This support enables us to keep working as we do – but we must maintain and build on it for every year to come. Support The Guardian from as little as $1 – and it only takes a minute. Thank you.
Source: https://www.theguardian.com/technology/2018/nov/18/how-can-you-tell-who-is-human-online-chatbots
Magic of Chatbot 