Then there are accents based on different geographic regions for English speakers within the U.S. Babbel identified 14 main groups of accents for American English, including New England, Pacific Northwest, Midwestern, Southern, New Orleans, East Coast, African American Vernacular, and Native American. Each of these accents has its own distinct vocabularies, syntax structures, and phonetic rules. For example, a Boston accent often drops r’s, which could lead the voice assistant to misunderstand common words, such as park, car, card, start, and color, creating user frustration, bad reviews, or failed purchases. 

If a voice assistant can’t complete its primary function of understanding the user, the customer will stop using the technology, maybe forever. The stakes become even higher when the voice assistant is a vital part of interactions with the world, such as a voice-enabled drive-thru at a QSR or a voice AI call center. A user can’t simply abandon these types of devices like they would if it was in-car or a smart TV. Instead, they have to suffer through not being understood while trying to complete the task, creating a frustrating user experience. 

Excluding specific portions of the population, even unintentionally though lack of training data, contributes to discrimination. Companies should carefully consider accents when embarking on their voice-first strategy so they can spread their values of inclusivity and also reach a larger target market. 

The challenges of training voice models with accents

Accents are much like another language, complete with their own vocabulary, syntax, and phonetic rules. The complexities of language differences make creating accent-agnostic speech recognition systems nearly as challenging as offering distinct languages. In some situations, creating a completely different model for the accent as if it were another language is the best option to ensure it is trained for all patterns that may arise. 

The data shows that creating voice assistants that understand accents is no minor feat, with even large companies falling behind. According to a study done with the Washington Post and Pulse Labs, Google Home and Amazon Alexa had a 30% accuracy difference between native speakers versus those with accents, with Spanish and Chinese being some of the most difficult accents for voice assistants to understand and Western U.S. and Southern U.S. being the easiest. Another study by Uswitch compared 30 different British accents with Cardiff, Wales and Glasgow, Scotland having the lowest accuracy rate while London had the highest. These studies show that it truly depends on the training data used as to which accents are understood by the voice assistant.

When embarking on a voice-first strategy, companies should consider these best practices for training a voice assistant to understand accents: 

• Have a diverse user base in training data
• Use conversational data 
• Start from the beginning 

Many companies fail at introducing enough accents into their training data in a conversational manner, relying on a Californian accent reading from newspapers like The Wall Street Journal as their primary data corpus instead of a variety of accents that map the different vocabularies and syntaxes that users speak in daily conversation. 

Instead, companies hoping to extend their product functionality and improve user experiences should introduce a diverse variety of accents into their training model so that it can learn as much as possible about different vocabularies, syntaxes, and phonetic rules. If only one accent is introduced, then the voice assistant won’t be able to deliver a positive user experience for a large portion of your customers.

Accented language is not something that can be added at a later point in the process. Training data and language models are key elements of building a voice assistant that will provide accurate results and exceptional user experiences. Companies need to be proactive about their voice assistants understanding accents, taking important steps from the very beginning of the project. It’s not an easy fix once halfway through the process or in the final steps. Oftentimes, it’s only after the voice assistant has been launched that companies start taking into account how big of a gap there is in understanding accents. 

Quick fixes, such as introducing gathered queries into the training model or adding specific vocabulary words into the dictionary, are only going to be a bandaid, patching a single facet instead of the larger pattern. Taking this approach, companies find they have to go back to the very beginning of the project and retrain their models to understand accents. Instead of getting caught in a voice assistant redo, carefully evaluate your approach from the beginning and introduce a diverse, conversational data set from the start. 

The importance of vocabulary for conversational AI

The material the voice actor uses to provide training data matters as much as the accent. Training models will learn and adapt to whatever vocabulary and syntax they are presented with, which means brands should also be aware of the reading material used to create the language corpus. For the best results, the content should be diverse and reflect how users speak casually and informally. 

If the content is from a source such as The Wall Street Journal or another newspaper where the writing is structured, grammatical, and formal, the voice assistant ultimately won’t understand how people speak in a conversational vernacular and create user frustration for aspects such as slang, contractions, and ungrammatical sentences. It also won’t teach the model vocabulary words that are unique to specific accents. Companies should get data from a wide variety of sources, even Twitter, to teach their training model how users speak in an everyday manner. 

Build, buy, or partner for accent-agnostic voice AI

Accents play a significant role in users’ experiences, which is why it’s so important for companies to carefully evaluate whether they want to build, buy, or partner for their voice assistant. Will the company that they partner with, build, or buy from understand the importance of accents and how to ensure a variety of them are introduced in training data?