From driving the world to dreaming it

From driving the world to dreaming it

Community

With Jeff Hawke, co-founder and CTO of Odyssey, at RAAIS 2026.

Air Street Press

and

Nathan Benaich

Jul 14, 2026

Article voiceover

0:00

-8:19

Audio playback is not supported on your browser. Please upgrade.

Jeff Hawke believes AI is missing a form of intelligence. Frontier research, as he sees it, runs on two macro bets: whether AI can bootstrap its own intelligence - the wager behind the labs chasing recursive self-improvement - or whether it can, in his words, “learn from the world directly.” Odyssey , where Hawke is co-founder and CTO, takes the second. The aim is to model the world as it is seen and acted on - “a representation that is richer than language,” built from raw sights and sounds rather than from concepts a human has already written down.

That was once a hard sell. When Odyssey pitched its seed round, he says, “basically no one understood world models,” and convincing people was “an uphill battle.” The category has since shifted underneath him: the argument flipped around November 2025, money and researchers flooded in, and at NeurIPS 2025 world models were, by his count, the field’s number one or two theme. Hawke came to the problem from self-driving, as a founding engineer at Wayve teaching cars to drive end-to-end - one of the few arenas, he says, where frontier AI has really been tested against reality.

Not every world model is a world model

The term is loose enough to mean almost anything, so Hawke was careful to pin it down. Odyssey uses it in the precise sense inherited from model-based reinforcement learning: a learned transition-dynamics model, the thing David Ha and Jürgen Schmidhuber named in their 2018 “World Models” paper. In plain terms, a model that learns how the world evolves and then samples possible futures one step at a time, conditioned on the actions you feed it. That is distinct from “spatial intelligence,” which models the appearance and structure of a scene (the pitch at World Labs and others); from behavior models, the decision-making brain of a self-driving car or a robot, which Hawke thinks is better named as such; and from the “proxy world models” some researchers argue are already latent inside LLMs. Odyssey’s version is what he calls a general-purpose neural simulator: an interactive stream of pixels that models physics, that you can talk to and that talks back, and that you can reach into and change. “I have many questions about what this means for future products,” he admitted. “I have very few good answers.”

Generality wins

Two principles guide what Odyssey builds, both carried out of self-driving. First, end-to-end learning, a simple model trained purely from data, “almost always wins.” It was an unpopular position in 2018 and is now the default in autonomy and in language alike. Second, “generality wins”: narrow models built for a single vertical rarely keep their lead for long. He couldn’t point to durable legal-specific foundation models, for instance, because “Claude just got better.” The ambition that follows is unsubtle. Odyssey wants to build “the GPT-3 of world models” - the InstructGPT-style moment when a category of model stops being a research demo and starts generating real commercial demand. Odyssey’s research splits into four problems, each with a shipped model against it.

Pixels that keep going…with sound too!

The foundation is autoregressive interactive pixels, embodied in Odyssey-2. A standard video model returns a fixed clip; this one streams frame after frame in real time, which is much harder, because error compounds as the model generates in sequence. It is also interactive, absorbing text prompts mid-stream and adjusting to them. Scaling helps in the usual way - a one-billion to a fifteen-billion-parameter jump lifts the benchmarks - but the real difficulty is keeping the interaction open-ended instead of narrowing it to a single domain.

In May 2026, Odyssey released Starchild-1, named for 2001: A Space Odyssey, which generates pixels and audio jointly in one coherent stream rather than dubbing sound onto finished video. It was harder than expected, Hawke said; to his knowledge no one else had shown one publicly. The obstacle is a clash of timescales - a single predicted video frame spans only “half a phoneme” of audio - so keeping the two coherent took a custom KV-cache design running on two clocks at once.

Shared state, not stitched video

The third problem is multiplayer: shared state across world models, which matters as much for a cell of robots working one environment as it does for a game. Odyssey’s Agora-1 demonstrates it, and Hawke ran it live. The audience pointed their phones at a QR code and played a fully generated game of GoldenEye, streamed off H100s “probably in Spain.” Under the hood it borrows a game engine’s split between rendering and simulation, except both halves are learned - a neural simulator and a neural renderer, trained together. People had said multiplayer world models “weren’t possible,” he noted, “and we felt it was worth disproving.”

Learning by being broken

The fourth strand carries the freshest idea in the talk. Almost everyone who pairs agents with world models uses the agent to get smarter inside the model. Odyssey’s PROWL turns that around: it uses a reinforcement-learning agent to improve the model itself. The reasoning is “garbage in, garbage out” - “your agent will never really be better than the quality of your learned environment model” - and yet, Hawke argued, almost no one had bothered to fix the model rather than the agent. PROWL drops an agent into a learned version of Minecraft and rewards it for finding the world model’s failure modes, then folds those failures into a curriculum that patches them. The wrinkle is that reinforcement learning is very good at cheating: left unleashed, the agent would “sit on the spot, spin the camera around at very high speed” to stall the model’s learning, so the team added a KL “leash” to keep it close to an agent that actually plays the game. The payoff was a clear lift in the base model’s performance - a world model that gets better by being systematically broken.

Still the GPT-2 era

For all of that, Hawke was disciplined about where the work sits. World models, he said, are at “the GPT-2 era”: the pre-ChatGPT stage where the outputs already look promising and the first commercial experiments are forming - he name-checked Jasper, the early GPT-3 copywriting startup - but mass adoption is still ahead. That is the gap between today and the “GPT-3 of world models” he wants to build. The honesty extends to limits. Asked from the floor whether a world model would have to encode general relativity and quantum mechanics, he declined the bait. Visual data is “by far the highest volume” and the right place to start, and these models earn their keep where conventional simulation struggles: for a precise definition of turbulent flow, “use CFD”; for crowds, contact, the messy texture of a scene, the neural simulator wins. Biology and harder physics are wanted, but, he conceded, nascent.

The economics are gentler than the hardware implies. The multiplayer demo ran on a single H100, “let’s call it $1.50 an hour” - close enough to a Netflix subscription that, by his estimate, $30 a month buys ten to fifteen hours of generated play.

Where this goes, on his telling, is convergence: as a visual world model takes on audio and, eventually, text, it meets the language models arriving from the other direction through VLMs, and at some point the two merge. None of that is solved, and he was candid that no one yet knows how. But the direction is set, and the market has caught up to the thesis that once needed an uphill seed pitch. On June 17, 2026, days after RAAIS, Odyssey raised a $310M Series B at a $1.45B valuation, led by Natural Capital with Amazon, AMD Ventures and GV. Air Street backed the company’s seed in 2024, on the bet that learning the world directly would become its own category of model. The GPT-2 era doesn’t last long.

Read more!

Previous

Next