A world model is a learned internal model of how an environment behaves — how it responds to an agent’s actions and what the agent would observe next — trained from past experience or recorded data. Once trained, it lets an agent simulate (“imagine”) trajectories inside the model itself, rather than only by acting in the real environment, which is what a model-free reinforcement-learning agent has to do for every training step.
Training a policy inside a world model instead of the real environment is attractive wherever real interaction is slow, expensive, or unsafe — such as physical robotics — but it shifts the burden onto the world model’s own accuracy: a policy that exploits errors in an inaccurate model can perform well “in imagination” while failing in the real environment the model was meant to stand in for.
Related papers
- Hafner, Yan, and Lillicrap, “Training Agents Inside of Scalable World Models”, arXiv:2509.24527 (2025) — trains an agent (Dreamer 4) entirely by reinforcement learning inside a learned Minecraft world model. Covered in 2026-w32.