ChatGPT could help passengers communicate better with AVs, study finds

Riding in an autonomous vehicle might have you missing friendly chit-chat with a driver. Would talking to ChatGPT help?

Researchers at Purdue University found that integrating AI-powered chatbots into AVs could help out in at least one way: making driverless vehicles better at taking orders from passengers. The study suggested that ChatGPT made it easier than the vehicle’s existing system for passengers to communicate things like being in a hurry or wanting a different temperature.

The researchers, who presented their findings for the first time last month, believe it’s one of the first experiments into how an autonomous vehicle can use large language models (LLMs) to interpret passenger commands.

“The conventional systems in our vehicles have a user interface design where you have to press buttons to convey what you want, or an audio recognition system that requires you to be very explicit when you speak so that your vehicle can understand you,” Ziran Wang, an assistant professor in Purdue’s Lyles School of Civil and Construction Engineering and leader of the study, said in the news release. “But the power of large language models is that they can more naturally understand all kinds of things you say. I don’t think any other existing system can do that.”

Experimentation: Researchers trained ChatGPT on prompts that included direct orders, like “drive faster,” as well as more vague comments like, “I feel a bit motion sick right now,” per the release. The tests largely were conducted at a proving ground in Indiana where the vehicles were able to operate at highway speeds.

“As ChatGPT learned how to respond to these commands, the researchers gave its large language models parameters to follow, requiring it to take into consideration traffic rules, road conditions, the weather, and other information detected by the vehicle’s sensors,” according to Purdue.

A vehicle equipped with Level 4 autonomy capabilities (meaning, no driver supervision is needed) then was given access to the models over the cloud. The LLMs worked through commands within the established parameters and provided instructions to the vehicle on how to respond.

Researchers reported that the study’s participants experienced less discomfort than is typical for passengers in a Level 4 AV that doesn’t have assistance from LLMs.

“Real-world experiments proved that the proposed system can comprehend human intentions at different intuition levels, ranging from direct commands like, ‘Can you drive faster?’ to indirect commands like, ‘I am really in a hurry now,’” researchers concluded.

Researchers noted that LLMs “hallucinating” is “an issue that must be addressed before vehicle manufacturers consider implementing large language models into AVs.” Further testing and regulatory approval would also be needed if LLMs were to be integrated into functions that control the vehicle.

As they continue to study the possibilities around this technology, researchers teased potential capabilities like LLMs in AVs communicating with each other at stop signs.

Integration: AI is being integrated into the auto industry in all kinds of ways—from helping to enable vehicle automation to changing the way we buy cars to being introduced into the vehicle’s cabin.

Now, automakers are starting to integrate chatbots into their vehicle lineups.

Last month, for example, Volkswagen of America announced it was introducing a subscription-based voice assistant tool featuring ChatGPT. The automaker said that one of the benefits is being able to speak to the vehicle more naturally.

VW-owned Audi is also bringing ChatGPT to its infotainment system to improve the vehicle’s voice control feature. The automaker teased at future possibilities.

“In the future, it will also be possible to ask questions about the vehicles, for example about the correct tire pressure,” according to Audi. “The use of large language models…makes this and many other use cases possible.”