Waymo World Model Sets New Standard for Autonomous Driving Simulation

On February 6, Waymo introduced the Waymo World Model, a frontier generative system designed to significantly advance autonomous driving simulation. Built to support safer and faster development of self-driving technology, the Waymo World Model establishes a new benchmark for large-scale, hyper-realistic virtual testing environments used in autonomous vehicle validation.

Waymo World Model built on advanced generative intelligence

The Waymo World Model is developed on Genie 3, the most advanced general-purpose world model created by Waymo. Genie 3 is capable of generating photorealistic, interactive 3D environments and has been adapted to meet the demanding requirements of autonomous driving simulation and safety validation.

Simulating rare and extreme driving conditions

By leveraging Genie 3’s extensive world knowledge, the Waymo World Model can recreate extremely rare scenarios that are nearly impossible to observe at scale in the real world. These include unusual weather events or unexpected encounters, allowing engineers to test how autonomous systems respond under extraordinary conditions.

High controllability and multi-sensor simulation

A defining feature of the Waymo World Model is its high level of controllability. Engineers can adjust driving inputs, traffic behavior, and scene layouts using simple language-based prompts, enabling rapid iteration and targeted testing across diverse driving situations. The model produces high-fidelity, multi-sensor outputs that closely mirror real-world perception systems. These outputs include synchronized camera and lidar data, ensuring simulations accurately reflect how the Waymo Driver interprets complex environments using its full sensor suite.

Transferring world knowledge from video to lidar

Genie 3’s training on an extremely large and diverse video dataset provides strong foundational world knowledge. With specialized training techniques, Waymo engineers are able to transfer this knowledge from 2D video into precise 3D lidar representations tailored to Waymo’s hardware, expanding simulation coverage beyond what fleet data alone can provide. The Waymo World Model can transform standard video footage, including content captured with regular cameras, into a multimodal simulation that shows exactly how the Waymo Driver would perceive the scene. Waymo has shared multiple demonstrations and technical details illustrating how this system operates and supports safer, more robust autonomous driving development.