The banality of deception

Some might object that the situation described in the Turing Test, where a machine tries to deceive us into believing it is human, doesn’t reflect our everyday experience with AI. It might happen in some specific, relatively rare situations – for instance with some bots on social media – but is not key to our most common uses and engagements with AI.

This, however, doesn’t mean that more subtle but still significant forms of deception are not at play in everyday interactions with AI technologies. Take, for instance, the case of voice assistants such as Apple’s Siri or Amazon’s Alexa. The choice to assign to voice assistants humanlike voices instead of synthetic-sounding voices, and to provide them with precise connotations of gender and even social and regional accent, are never made by chance. They derive from considerations made by Amazon and Apple on how users will react to different kinds of voices, and how such reactions can help achieve specific outcomes – for instance, encouraging users to integrate these tools into their domestic environments and everyday lives. Although users of Siri and Alexa are perfectly able to understand that these are computer programs and not real persons, the gendered voice of virtual assistants activates mechanisms of representation by which users imagine a source for the voice – and, subsequently, a stable character with which to interact, even if within relatively strictly boundaries. The gender, class and race clues embedded in the synthetic voices create the psychological and social conditions for projecting an identity and, to some extent, a personality onto the virtual assistant.

We can describe these forms of apparently inoffensive deception with the term “banal deception.” Banal deception entails mundane, everyday situations in which technologies and devices mobilize specific elements of user’s perception and psychology – such as the all-too-human tendency to attribute agency to things or personality to voices. The deception of Alexa and Siri is “banal” because it has to do with situations that are immersed in our everyday life and that we don’t even perceive as deceptive. The ordinary and mundane character of banal deception makes it unnoticeable but more consequential since it helps these technologies enter the deepest layers of our everyday habits and behaviours.

In contrast to outright, ‘stronger’ forms of deception, banal deception can have, at least potentially, some value for the user. For example, the fact that users respond socially to voice assistants brings an array of pragmatic benefits: it makes it easier to use these tools and creates space for playful interaction and emotional reward. For companies and designers, moreover, the fact that banal deception is not perceived as such has an advantage at a commercial level, since the user maintains the illusion of having full control of the experience.

Examples of banal deception in contemporary AI systems are manifold. Voice assistants mobilize gender and class stereotypes through the accent and the characterization of their voice, aiming at specific responses from users. Similar mechanisms are activated by chatbots in written communications, as shown by studies of users’ responses to chatbots using emojis. In companion robots and chatbots, a sense of cuteness is created through specific aspects of their design, helping to activate mechanisms of empathy that provide an emotional reward to users. On social media, bots programmed to impersonate fictional characters populate accounts that disclose their mechanical nature, but still stimulate engagement from users.

The ordinary and mundane character of banal deception makes it unnoticeable but more consequential since it helps these technologies enter the deepest layers of our everyday habits and behaviours.

The notion of banal deception helps identity that deception is not an exception in AI; on the contrary, it plays a fundamental role in AI technologies that are programmed to interact with humans. One of the implications of this is that AI system cannot be understood only by examining the internal functioning of the machine: one needs to inquire also the social, psychological, and cultural dynamics that AI activates when it engages users.

 Public discussions of AI usually emphasise the evolution of the technology, which has become more and more sophisticated and performative. But in fact, the functioning of AI and its impact does not depend only on technical features. As Alan Turing intuited when he proposed his Test, the outcome of the interaction between a machine and a human depends on the characteristics of the machine as well as on the characteristics of the humans who participate in the interaction. For instance, if a judge of the Turing Test has a strong background in computer science, she or he will likely give a different assessment than someone who hasn’t much knowledge or experience with this technology. The result is that developments in AI have to do with technical advances as well as with the development of a range of features and strategies by which AI developers mobilise perceptive mechanisms, habits, and social understandings of human users in order to make these technologies achieve the desired effect.

The questions posed in the debate about Perseverance’s Twitter account should thus be reformulated. What we should ask, instead, is how to develop more informed and self-aware relationships with technologies that are programmed to take advantage of our liability to be deceived. It might sound paradoxical, but to better comprehend AI we need first to better comprehend ourselves. Contemporary AI technologies constantly mobilize mechanisms such as empathy, stereotyping, and social habits. To understand these technologies more deeply, and to fully appreciate the relationship we are building with them, we need to interrogate how such mechanisms work and which part deception plays in our interaction with “intelligent” machines.