Three engineers connected advanced language models like OpenAI’s GPT-6 Astra to a 2024 Toyota Corolla, enabling the AI to navigate a drive-thru and order food, marking a novel experiment in AI’s physical world understanding.
- GPT-6 Astra successfully piloted a Toyota Corolla to an In-N-Out drive-thru.
- Language models show emerging physical reasoning abilities beyond digital tasks.
- Experts highlight safety concerns and the need for further research in robotic applications.
What happened
A trio of engineers set up a 2024 Toyota Corolla with cameras and a power steering interface linked to a laptop running OpenAI’s GPT-6 Astra. Normally tasked with text and code generation, the model was asked to take control of the car and drive it to an In-N-Out Burger drive-thru near the Bay Area. A safety driver remained attentive throughout the test, ready to intervene if necessary.
Despite GPT-6 Astra being a general-purpose language model with no specialized driving training, it managed to navigate the car through the complex real-world environment of a drive-thru queue to place the order. Other models such as SpaceXAI’s Grok initially refused but were coaxed via careful prompting, with Astra ultimately succeeding.
Why it matters
This experiment highlights that sophisticated language models are beginning to develop rudimentary physical world understanding, a significant step beyond their traditional digital and text-based applications. Such capabilities could broaden AI’s role into robotics, autonomous vehicles, and environments requiring spatial reasoning.
However, the attempt is also a reminder of the challenges and risks involved. AI models like GPT-6 Astra were not designed with direct control over heavy machinery in mind, and despite success, human oversight was essential. Continued development in physical reasoning benchmarks and safety protocols is critical before wider deployment.
What to watch next
The frontier of physical reasoning remains a priority for AI researchers aiming to unlock new applications in home robotics, autonomous vehicles, and real-world interaction. Benchmarks like Humanity’s Sixth Sense are emerging to measure how intuitively AI can understand and react to physical scenes and tasks without exhaustive programming.
Future progress will depend on improving spatial awareness and balancing safety with autonomy. Industry players and startups focused on physical AI reasoning will likely drive innovations, while regulatory aspects and ethical considerations around putting general AI models in control of real-world systems will evolve alongside the technology.