AI voices deceive humans, but the brain’s reactions differ

Summary: Humans have difficulty distinguishing between human and AI-generated voices, and can only correctly identify them about half of the time. Despite this, brain scans showed different neural responses to human and AI voices: human voices activate areas related to memory and empathy, while AI voices activate regions for error detection and attention regulation.

These findings highlight both the challenges and potential of advanced AI voice technology. Further research will examine how personality traits affect the ability to recognize the origin of voices.

Important facts:

1. Identification struggles: Participants correctly recognized human voices in 56% of cases and AI voices in 50.5% of cases.
2. Neuronal reactions: Human voices activated memory and empathy areas; AI voices triggered error detection and attention regulation.
3. Perceptual distortion: Neutral voices were often perceived as AI, while happy voices were considered human.

Source: FENS

Humans are not particularly good at distinguishing between human voices and voices generated by artificial intelligence (AI). But our brains react differently to human and AI voices, according to a study presented today (Tuesday) at the Forum 2024 of the Federation of European Neuroscience Societies (FENS).

The study was presented by doctoral student Christine Skjegstad and conducted by Ms Skjegstad and Professor Sascha Frühholz, both from the Department of Psychology, University of Oslo (UiO), Norway.

Ms Skjegstad said: “We already know that AI-generated voices have advanced to the point where they are almost indistinguishable from real human voices. It is now possible to clone a person’s voice from just a few seconds of recording and fraudsters have used this technology to impersonate a loved one in need and trick victims into transferring money.

“While machine learning experts are developing technological solutions to recognize AI voices, much less is known about how the human brain responds to these voices.”

The study involved 43 people who were asked to listen to human and AI-generated voices expressing five different emotions: neutral, angry, fear, happy, joy. They were asked to identify the voices as synthetic or natural while their brains were scanned using functional magnetic resonance imaging (fMRI).

fMRI can detect changes in blood flow in the brain that indicate which parts of the brain are active. Participants were also asked to rate the characteristics of the voices they heard in terms of naturalness, trustworthiness and authenticity.

Participants correctly identified human voices only 56% of the time and AI voices 50.5% of the time. This means that they were equally bad at recognizing both voice types.

People were more likely to identify a “neutral” AI voice as AI (75% compared to 23% who could identify a neutral human voice as human), suggesting that people assume neutral voices are more similar to AI.

Neutral female AI voices were correctly identified more often than neutral male AI voices. For happy human voices, the correct recognition rate was 78%, compared to only 32% for happy AI voices, suggesting that humans perceive happiness as more human.

Both AI and human neutral voices were perceived as the least natural, trustworthy, and authentic, while happy human voices were perceived as the most natural, trustworthy, and authentic.

However, when examining brain images, researchers found that human voices elicited stronger responses in areas of the brain associated with memory (right hippocampus) and empathy (right inferior frontal gyrus).

AI voices elicited stronger responses in areas related to error detection (right anterior middle cingulate cortex) and attention regulation (right dorsolateral prefrontal cortex).

Ms Skjegstad said: “My research shows that we cannot tell very accurately whether a voice is human or generated by an AI. Participants also often expressed how difficult it was for them to tell the difference between the voices. This suggests that current AI voice technology can mimic human voices to such an extent that it is difficult for humans to reliably tell them apart.

“The results also suggest a perceptual bias, where neutral voices were more likely to be perceived as being generated by AI and happy voices were more likely to be perceived as more human, regardless of whether they actually were. This was particularly the case for neutral female AI voices, which may be due to our familiarity with female voice assistants like Siri and Alexa.

“Although we are not very good at distinguishing human voices from AI voices, there seems to be a difference in the brain’s response. AI voices can elicit heightened attention, while human voices can elicit a sense of connection.”

The researchers now want to investigate whether personality traits such as extraversion or empathy lead people to perceive the differences between human and AI voices more or less sensitively.

Professor Richard Roche is Chair of the FENS Forum Communications Committee and Deputy Head of the Department of Psychology at Maynooth University, Maynooth, County Kildare, Ireland and was not involved in the research.

He said: “Studying the brain’s responses to AI voices is critical as this technology continues to advance. This research will help us understand the potential cognitive and social impacts of AI voice technology, which can potentially help support policy and ethical guidance.

“The risk of this technology being used to cheat and deceive people is obvious. However, there are also potential benefits, such as providing voice replacements for people who have lost their natural voice. AI voices could also be used in therapy for some mental illnesses.”

About this research news from the fields of AI and neuroscience

Author: Kerry Noble
Source: FENS
Contact: Kerry Noble – FENS
Picture: The image is from Neuroscience News.

Original research: The results will be presented at the FENS Forum 2024