During each stage of developing their voice assistant, the Mercedes-Benz team kept their focus on two important elements, their customers and their brand. Staying true to the Mercedes-Benz value proposition meant knowing who they are as a company and making sure their voice assistant communicated those values and attributes at every level, especially when it came to adding personality, tone, and humor to the voice assistant.

In the end, the goal was simple: Get people to interact with their cars with simple voice commands and allow them to be able to ask for anything in a conversational manner, just like they’d talk to another person in the car.

Building a seamless, frictionless in-car voice experience

The Mercedes-Benz mission has always been to provide people with the most comfortable and safe driving experience possible. The development of a voice-enabled infotainment system was a natural extension of that vision. If the team was going to add an in-car voice assistant, then it had to deliver the best experience possible. 

To achieve this level of conversational interaction and accuracy, the team set three specific goals:

1. Allow drivers to talk as naturally as they would to another person and be able to say things naturally like, “I’m hungry” or “I’m cold.”
2. Support more types of queries—specifically cloud-enabled services—such as buying concert tickets or pre-ordering coffee.
3. Integrate the voice assistant even more naturally in the overall experience in the car, so users can switch smoothly back and forth between touch controls and voice commands.

For the immediate future, the team focused on moving from a world where drivers have knobs and buttons for everything, to providing an alternate input source for complex functionality and making those capabilities easily discoverable. 

Based on the knowledge that people are getting used to using voice assistants in their homes and on their phones, they set out to make the experience similar, but also relatable to the in-car environment. They knew that their customers want to have an easy interaction with the car in a way that helps them on a daily basis—by making routines easier, drives safer, and everything faster.

Overcoming the mindset of rigid voice commands

Developing a voice assistant for any product or application is not an easy task. With the help of our Houndify engineers, the team at Mercedes-Benz was able to extend the voice assistant to the cloud while helping their users change ingrained behaviors around addressing and interacting with voice assistants in a very rigid way.

Changing how we talk to voice assistants—shifting from barking a specific set of commands to having a natural conversation—was a challenge. And the change needed to start within the engineering team.

Making sure that every driver has the same positive experience requires the assistant to understand accents, dialects, and common languages used by drivers in the countries where their cars are sold. Even within the U.S., the multitude of accents and pronunciations posed a unique challenge. 

Overcoming initial hurdles and creating a voice user interface that closely resembles that of human-to-human interaction is only the first step for the team at Mercedes-Benz. They are constantly looking to the future and for ways to improve the driving experience with voice technology. 

The Mercedes Voice Assistant – Powered by Houndify

The partnership between Mercedes-Benz and SoundHound Inc. opened the door to a unique, powerful, and exceptionally-helpful cloud-based user experience for drivers.

Houndify’s extensive library of hundreds of content domains helped the MBUX voice team reach their goals faster, eliminating the need for them to start from scratch to build out all the responses to queries themselves, while still giving them the flexibility to customize any response.

Houndify’s hybrid engine and domain libraries enabled the team to create an intuitive and truly conversational voice assistant and has extended the “Hey Mercedes” experience from only controlling in-car functionality (like windows and air-conditioning) to delivering results for external things like sports scores, general knowledge questions, weather, and so much more.

From the beginning, the teams were focused on communicating the unique Mercedes-Benz brand with a voice assistant that would eventually come to be like a friend. Houndify’s proprietary Speech-to-Meaning ® technology combines the traditional two-step ASR and NLU models into one integrated action that processes speech and delivers results in real time, giving the illusion of talking to a real person.

Building the MBUX was a multi-year project with many components. Once the team started working with SoundHound Inc., the cloud-based component of the voice user interface took two years to implement—a fairly short sprint in an otherwise long journey. The MBUX infotainment system as drivers experience it now is only the foundation for their vision of the future.

Collaboration and teamwork make all the difference

The mantra of voice user interface (VUI) design is to build, test, iterate, test, iterate, and then iterate some more. The keys to making that process work is agility, good user data, and engineering expertise to execute the vision. The partnership between Mercedes-Benz and SoundHound Inc. had all three.