How Waabi is combining reality and simulations to test AVs ahead of driverless launch

Simulation, meet reality.

That’s the idea behind mixed reality testing, or MRT, a method AV trucking startup Waabi is employing as it works to launch fully driverless operations later this year.

MRT pairs AI-generated simulations with testing on a closed-course track. The strategy, according to Waabi CEO and founder Raquel Urtasun, is akin to outfitting a self-driving truck with augmented reality goggles.

“MRT is one of the coolest technologies ever, and it’s actually an extremely important technology as it relates to safety,” she told Tech Brew.

On the track: Urtasun explained that over the last century, closed-course testing has been the primary way companies test new vehicles, whether they’re human-operated or autonomous. She likened the process to doing movie stunts; it involves lots of time, resources, and coordination.

It’s also difficult to precisely repeat such physical tests, and there’s a limited diversity of scenarios you can recreate in such an environment, Urtasun wrote in a blog post.

Although equipment like dummy props addresses some of these challenges, “they are expensive and fundamentally unrealistic,” she wrote. And some scenarios—like crashes or impaired driving—aren’t safe, ethical, or practical to test in the real world.

Urtasun cited the example of construction: Although it’s a frequent roadway scenario, it’s one that varies quite a bit. An AV company could try to account for every real-world construction possibility out there and then test its tech on a closed course.

“Suddenly you can…train the system to do things that otherwise would take you billions or trillions of miles to even see once,” Urtasun said. “How many times have you seen somebody on the highway [driving] in the opposite direction?”

Mix it up: Simulation has become a common tool AV companies use to test their tech. Waabi previously introduced a realism score showing that its simulator, Waabi World, is 99.7% accurate. Waabi World uses GenAI to come up with new scenarios, including some “that may have never happened in the real world,” Tech Brew previously reported.

“If we were able to bring the simulator to the physical world, then we would have the ability to really test all the safety-critical situations, be reproducible, be at scale, with no consequences,” Urtasun said. “Suddenly you open the possibilities to really test anything that you can think of.”

Enter MRT, which Waabi started using about two years ago and which has enabled it to run thousands of closed-course tests per day.

Using MRT, Urtasun said, was crucial to Waabi achieving the milestone of becoming “feature complete,” meaning its AV system is capable of going driverless.

As Waabi’s simulation system generates scenarios, Urtasun explained in her blog post, “the real physical sensor readings are modified instantaneously so the Waabi driver can react to the blend of real and virtual elements while driving in the physical world.”

“Traffic jams can materialize instantly, motorcycles can weave between lanes, crowds of pedestrians can jaywalk unpredictably, and children can dart into the street from behind parked cars,” she wrote.

Waabi, whose autonomous trucks operate on public roads in Texas with human safety drivers on board, has partnerships with Uber Freight and Volvo Autonomous Solutions.

Waabi leaders view MRT as a way to build trust with customers and regulators, and to improve its capital efficiency, according to Urtasun. And she noted that the tech could be applied to other forms of physical AI, like robots.

“We definitely plan to do more than just self-driving trucks in the future,” she told us. “But right now we are very, very focused on launching the product, which is end of year, and then many more things will follow.”