Researchers want to reduce the harm to multicultural users of voice assistants

Users of voice assistants such as Siri, Alexa or Google Assistant know the frustration of being misunderstood by a machine.

But for people without an American accent, such misunderstandings can be more than just annoying, they can be downright dangerous, say researchers at the Human-Computer Interaction Institute (HCII) at Carnegie Mellon University’s School of Computer Science.

In a new study published in Proceedings of the CHI Conference on Human Factors in Computing SystemsHCII PhD student Kimi Wenzel and Associate Professor Geoff Kaufman identified six harms caused by errors in voice assistants and developed strategies to reduce them. Their work won the Best Paper Award at the Association for Computing Machinery’s Human Factors in Computing Systems (CHI 2024) conference.

“This article is part of a larger research project in our lab to document and understand the impact of biases embedded in technology,” Kaufman said.

White Americans are overrepresented in most datasets used to train voice assistants, and studies have shown that these assistants are much more likely to misinterpret or misunderstand black speakers and people with accents or dialects that differ from standard American English.

Previous researchers viewed this issue as a technical problem to overcome rather than a bug that impacts the user, Kaufman said. But when language is misunderstood, whether by a human or a machine, it can be perceived as a microaggression.

“It can impact self-esteem or a sense of belonging,” Kaufman said.

In a controlled experiment last year, Kaufman and Wenzel examined the effects of a voice assistant’s error rate on white and black volunteers. Black people with high error rates showed higher levels of self-awareness, lower self-esteem and less positive attitudes toward the technology than black people with low error rates. This response was not observed in white people, regardless of error rate.

“We hypothesize that these experiences are intensified and that people suffer more from the negative consequences because black people are more frequently confronted with misunderstandings or have more everyday experiences with racism,” said Wenzel.

In the latest study, Wenzel and Kaufman surveyed 16 volunteers who had experienced issues with voice assistants. They found six potential harms that can result from seemingly harmless voice assistant errors. These included emotional harm and cultural or identity-related harm from microaggressions.

They also considered relationship damage, when an error leads to interpersonal conflict. For example, a voice assistant might make a calendar entry with the wrong time for a meeting or incorrectly route a call.

Other disadvantages include paying the same price as others for a technology even though it doesn’t work as well for you, and having to make extra efforts to make the technology work, such as changing your accent.

A sixth harm is physical danger.

“Speech technologies are not only used as simple voice assistants in smartphones,” says Wenzel. “They are also increasingly being used in more serious contexts, for example in medical transcription.”

Speech technologies are also being used in conjunction with car navigation systems, “and there is a lot at stake there,” Wenzel added.

One person interviewed for the study reported their own hair-raising experience with a voice-controlled navigation system. “A lot of times I feel like I’m saying things very clearly and loudly, but it still doesn’t understand me. And I don’t know what’s going on. And I don’t know where I’m going. So it’s just this frustrating experience and very dangerous and confusing.”

The ultimate solution is to eliminate bias in voice technologies, but creating data sets that represent the full range of human variation is a difficult task, Wenzel said. So she and Kaufman talked to participants about what voice assistants could say to their users to mitigate those harms.

One strategy they identified to fix the bug is blame shifting—that is, not a simple apology, but an explanation that describes the bug without blaming the user.

Wenzel and Kaufmann also suggest that language technologies become more culturally sensitive. Combating cultural damage is limited to some extent by technology, but a simple but profound measure would be to expand the database of proper names.

“Misrecognition of non-English names is a persistent problem in many language technologies,” the researchers noted in the article.

Numerous social psychology studies have shown that self-affirmation – a statement about a person’s values ​​or beliefs – can be protective when their identity is threatened, Kaufman said. He and Wenzel are looking at ways that voice assistants can build affirmations into their conversations with users, preferably in a way that is not obvious to the user. Wenzel is currently testing some of these affirmations in a follow-up study.

Brevity is key in all of these conversational interventions. After all, people often use voice technologies in the hope of being more efficient or being able to work hands-free. Adding messages into the conversation tends to work against that goal.

“This is a design challenge for us: How can we emphasize that the fault lies with the technology and not the user? How can we make this emphasis as clear as possible with as few words as possible?” Wenzel said. “Right now the technology says ‘sorry,’ but we think it should be more than that.”