Google's latest Agent technology will drive the next generation of in-car AI assistant for Mercedes.

The Mercedes MBUX

View pictures in App save up to 80% data.

Mercedes-Benz

On Monday, Google Cloud unveiled its latest Automotive AI Agent, with the Mercedes-Benz CLA set to be the first vehicle model to integrate this technology later this year. This new Agent will enhance the capabilities of Mercedes’ MBUX Virtual Assistant, allowing for a broader range of conversational interactions with passengers in the car.

We got our first look at Mercedes’ next generation assistant a year ago, at CES 2024, though the company did not reveal which large language model underpinned its capabilities at the time. This new assistant differs from the existing MBUX that can activate around two dozen in-car commands and provide information sourced from ChatGPT and Bing. While the current generation assistant can be activated by stating “Hey, Mercedes,” it functions more like Siri or Google Assistant than ChatGPT’s Advanced Voice Mode, offering static responses rather than conversational replies.

Google’s Agent is built atop the Gemini LLM using Vertex AI and is geared specifically to “allow automakers to create highly personalized and intuitive in-car agents that go beyond current vehicle voice control,” per the company’s announcement post. The Agent supports both multimodal and multilingual inputs, as well as can provide answers to follow-up questions. In Google’s example, the AI will be able to tell drivers if there are any Italian restaurants nearby, then offer up reviews of the establishment and even tell you what the most popular dish there is. The system is reportedly robust enough to handle multi-turn dialog with users and remember details from previous conversations.

The new MBUX assistant will reportedly pull “fresh and factual information” from Google Maps in near real time to offer “comprehensive and personalized information” about more than 250 million points of interest worldwide and current traffic conditions. “At Mercedes-Benz, we seek to offer our customers an exceptional digital experience,” said Ola Källenius, CEO of Mercedes-Benz Group AG, in a press statement. “Our partnership with Google Cloud will further enhance in-car navigation, combining sophisticated location data with generative AI.”

Mercedes intends to introduce the new MBUX assistant to more models in the future, although it has not yet disclosed which specific models will feature the integration.