← All articles
World models

Video generation models are not world simulators

Sora's time-consistency looks like physics, but vision is a lower-dimensional projection. A model trained on what seems to fit learns the shadow, not the world casting it.

A snow globe holding a tiny city at night.

The performance of Sora is no news. It is obvious to the naked eye how impressive it is, and you have probably seen the feats already. What caught my attention was not the video. It was the framing in the official technical report: video generation models as world simulators. Are they, though?

A world simulator requires a world model, a set of explicit rules and principles that govern the world. We cannot simulate physics without Newton's laws. And we cannot simulate Newton's laws just by mimicking their visual effects. The effect and the cause are not the same object. One you can copy; the other you have to understand.

The most important breakthrough in Sora is not resolution. It is time-consistency. An object can be obstructed for a few frames, and when it reappears it is the same object as before. Earlier models rarely managed this. Things would vanish forever behind an unfortunate obstruction. Because the new behavior looks so much like the world obeying spatial and temporal rules, it is tempting to pin it on a latent world model driving the video to stay coherent. Hence the title of the report.

I argue that the complexity of a world model cannot come out of image analysis alone. Vision, like language, is a lower-dimensional projection. A sphere casts a circular shadow, and from the circle you can only estimate the sphere. If we asked Plato, the conclusion would be quick: the prisoners in the cave study shadows, and these models do the same. They learn from our projection of the world, not from the world itself.

This is not a claim that AI is forever stuck at the surface. But the current architectures, trained on discontinuous patches by the criterion of what seems to fit, are weak candidates for true simulation. They can produce images that look right, and sentences that sound intelligent. For anything deeper, the depth has to live in the fabric of the model, not in the shadow it was fitted to.

Work with me

Want to talk it through?

If your company is working out where AI fits, the first conversation is free. A few short questions, and the reply comes from me within 24 hours on working days.