Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

The next frontier after language models is world models, and learning to predict the next state of a world. Hear from our researchers how these models really work at a low-level and the mountain we're climbing. A general-purpose world simulator (AKA sci-fi) is now in sight.

Odyssey

33,634 subscribers

25,871 views • 7 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The most clearest and crisp explanation, I've ever heard, of how large language models compress and capture a "world-model" in their weights simply by learning to predict the next word accurately. Furthermore, how the raw power of these base models can then be tamed by teaching them to follow instructions from humans. Source:

The most clearest and crisp explanation, I've ever heard, of how large language models compress and capture a "world-model" in their weights simply by learning to predict the next word accurately. Furthermore, how the raw power of these base models can then be tamed by teaching them to follow instructions from humans. Source:

Zain

970,967 views • 2 years ago

Introducing General World Models. We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models. Learn more:

Introducing General World Models. We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models. Learn more:

Runway

714,292 views • 2 years ago

Today is a big day at Odyssey. We're sharing Explorer, our first generative world model. We think world models are the next frontier for AI, enabling wonderful new things. To help shape this, Ed Catmull, the iconic founder of Pixar, is joining our board and investing.

Today is a big day at Odyssey. We're sharing Explorer, our first generative world model. We think world models are the next frontier for AI, enabling wonderful new things. To help shape this, Ed Catmull, the iconic founder of Pixar, is joining our board and investing.

Odyssey

75,278 views • 1 year ago

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.

abdel

36,685 views • 1 month ago

Demis Hassabis says world models are his longest standing passion and explains benefits vs. language models: ▫️ “I think language models are able to understand a lot about the world. More than we expected because language is actually probably richer than we thought. But there's still a lot about the spatial dynamics of the world, spatial awareness and the physical context we're in — and how that works mechanically — that is hard to describe in words and isn't generally described in corpuses of words. Alot of this is allied to learning from experience. There's a lot of things which you can't really describe something. You have to just experience it. Maybe the senses and so on are very hard to put into words. Whether that's motor angles and smell and these kinds of senses, it's very difficult to describe that in any kind of language.”▫️ This is what Demis and Google Deepmind is trying to solve with Genie. He also says that the video models (Veo) will play a part in training the world models and this is all key for AI robotics.

Demis Hassabis says world models are his longest standing passion and explains benefits vs. language models: ▫️ “I think language models are able to understand a lot about the world. More than we expected because language is actually probably richer than we thought. But there's still a lot about the spatial dynamics of the world, spatial awareness and the physical context we're in — and how that works mechanically — that is hard to describe in words and isn't generally described in corpuses of words. Alot of this is allied to learning from experience. There's a lot of things which you can't really describe something. You have to just experience it. Maybe the senses and so on are very hard to put into words. Whether that's motor angles and smell and these kinds of senses, it's very difficult to describe that in any kind of language.”▫️ This is what Demis and Google Deepmind is trying to solve with Genie. He also says that the video models (Veo) will play a part in training the world models and this is all key for AI robotics.

Bearly AI

123,703 views • 7 months ago

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

RoboPapers

27,567 views • 5 months ago

Demis on why world models are his longest standing passion and explains benefits vs. language models: ▫️ “I think language models are able to understand a lot about the world. More than we expected because language is actually probably richer than we thought. But there's still a lot about the spatial dynamics of the world, spatial awareness and the physical context we're in — and how that works mechanically — that is hard to describe in words and isn't generally described in corpuses of words. Alot of this is allied to learning from experience. There's a lot of things which you can't really describe something. You have to just experience it. Maybe the senses and so on are very hard to put into words. Whether that's motor angles and smell and these kinds of senses, it's very difficult to describe that in any kind of language.”▫️

Demis on why world models are his longest standing passion and explains benefits vs. language models: ▫️ “I think language models are able to understand a lot about the world. More than we expected because language is actually probably richer than we thought. But there's still a lot about the spatial dynamics of the world, spatial awareness and the physical context we're in — and how that works mechanically — that is hard to describe in words and isn't generally described in corpuses of words. Alot of this is allied to learning from experience. There's a lot of things which you can't really describe something. You have to just experience it. Maybe the senses and so on are very hard to put into words. Whether that's motor angles and smell and these kinds of senses, it's very difficult to describe that in any kind of language.”▫️

Bearly AI

78,545 views • 1 month ago

We've been working closely with NVIDIA to build the next generation of frontier models, which we debut today: our family of General World Models. We are thrilled to deepen our partnership with Jensen and his team. To celebrate, Jensen sent us a special holiday message. Hope you enjoy 🐶🎄

We've been working closely with NVIDIA to build the next generation of frontier models, which we debut today: our family of General World Models. We are thrilled to deepen our partnership with Jensen and his team. To celebrate, Jensen sent us a special holiday message. Hope you enjoy 🐶🎄

Cristóbal Valenzuela

249,899 views • 7 months ago

It’s time to go beyond language models. Introducing Odyssey-2 Max, our most powerful world model yet. It materially advances the SOTA in physical accuracy. This is a big step toward models that simulate and interact with the world in real time. A new intelligence entirely!

It’s time to go beyond language models. Introducing Odyssey-2 Max, our most powerful world model yet. It materially advances the SOTA in physical accuracy. This is a big step toward models that simulate and interact with the world in real time. A new intelligence entirely!

Odyssey

2,183,089 views • 2 months ago

Real-time World Models are the next AI frontier. Today, we're taking the first step towards this reality: our early preview lets you experience worlds generated in real-time, running on our global low-latency infrastructure. Try it now:

Real-time World Models are the next AI frontier. Today, we're taking the first step towards this reality: our early preview lets you experience worlds generated in real-time, running on our global low-latency infrastructure. Try it now:

reactor

10,295,975 views • 2 months ago

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

AK

52,644 views • 2 years ago

Introducing Omnia Alpha. Audio-driven generation with full control over camera, motion, and background. Built to cross the uncanny valley without sacrificing creative control. This release is part of a larger vision. The future of world models is open and local. We're exploring how to bring our upcoming models to consumer hardware. America needs a leader in open source world models, and we're building toward that. Full version coming soon. Until then, show us what you can do with Omnia. Available now.

Introducing Omnia Alpha. Audio-driven generation with full control over camera, motion, and background. Built to cross the uncanny valley without sacrificing creative control. This release is part of a larger vision. The future of world models is open and local. We're exploring how to bring our upcoming models to consumer hardware. America needs a leader in open source world models, and we're building toward that. Full version coming soon. Until then, show us what you can do with Omnia. Available now.

Hedra

3,105,305 views • 5 months ago

Watch "Endless experimentation: Building AI models in the wild." This is a 50-minute, MIT lecture about problems when deploying models and LLMs in the real world and how to prepare to solve them. This is a great lecture for those building production machine learning.

Watch "Endless experimentation: Building AI models in the wild." This is a 50-minute, MIT lecture about problems when deploying models and LLMs in the real world and how to prepare to solve them. This is a great lecture for those building production machine learning.

Santiago

117,204 views • 2 years ago

Punchline: World models == VQA (about the future)! Planning with world models can be powerful for robotics/control. But most world models are video generators trained to predict everything, including irrelevant pixels and distractions. We ask - what if a world model only predicted the semantic information necessary for decision-making? Introducing Semantic World Models (SWM). Given an observation and an action sequence, SWMs cast modeling as answering textual questions about the future outcome resulting from the actions. Recasting world modeling as a VQA problem lets us directly leverage the pretrained knowledge and machinery of VLMs for generalizable modeling. We had a lot of fun thinking about how this work helps connect these two seemingly very different fields of study - VLMs and world models! 🧵(1/6) Paper: Fun demo:

Punchline: World models == VQA (about the future)! Planning with world models can be powerful for robotics/control. But most world models are video generators trained to predict everything, including irrelevant pixels and distractions. We ask - what if a world model only predicted the semantic information necessary for decision-making? Introducing Semantic World Models (SWM). Given an observation and an action sequence, SWMs cast modeling as answering textual questions about the future outcome resulting from the actions. Recasting world modeling as a VQA problem lets us directly leverage the pretrained knowledge and machinery of VLMs for generalizable modeling. We had a lot of fun thinking about how this work helps connect these two seemingly very different fields of study - VLMs and world models! 🧵(1/6) Paper: Fun demo:

Abhishek Gupta

61,208 views • 8 months ago

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 views • 1 year ago

Demis Hassabis on the limit in today’s AI: language can describe the world, but it cannot contain it - and why "World Models" are his "longest standing passion". Language models absorbed far more structure about reality from text than many researchers expected, because human language quietly carries physics, psychology, culture, tools, plans, and cause-and-effect. But text is still a compressed residue of experience, not experience itself. A sentence can say a cup falls from a table, yet it does not fully encode weight, grip, balance, friction, timing, sound, surprise, or the tiny motor corrections a body makes before it even notices them. The world is not only made of facts that can be named; it is made of constraints that have to be lived through, touched, predicted, violated, and repaired. That is why world models matter. They aim to learn the hidden grammar of physical reality: how objects persist, how forces unfold, how space changes when an agent moves, and how action creates feedback. Language models can often reason about the world because people have written so much about it. World models try to learn what the world is like before it becomes words. The difference is exactly what matters because intelligence is not just answering well; it is knowing what would happen next if you moved, reached, pushed, smelled, slipped, or failed. A mind trained only on descriptions may become brilliant at explanation. A mind trained on experience may become better at consequence. --- Full video from "Google DeepMind" and "Hannah Fry" YT channel (link in comment)

Demis Hassabis on the limit in today’s AI: language can describe the world, but it cannot contain it - and why "World Models" are his "longest standing passion". Language models absorbed far more structure about reality from text than many researchers expected, because human language quietly carries physics, psychology, culture, tools, plans, and cause-and-effect. But text is still a compressed residue of experience, not experience itself. A sentence can say a cup falls from a table, yet it does not fully encode weight, grip, balance, friction, timing, sound, surprise, or the tiny motor corrections a body makes before it even notices them. The world is not only made of facts that can be named; it is made of constraints that have to be lived through, touched, predicted, violated, and repaired. That is why world models matter. They aim to learn the hidden grammar of physical reality: how objects persist, how forces unfold, how space changes when an agent moves, and how action creates feedback. Language models can often reason about the world because people have written so much about it. World models try to learn what the world is like before it becomes words. The difference is exactly what matters because intelligence is not just answering well; it is knowing what would happen next if you moved, reached, pushed, smelled, slipped, or failed. A mind trained only on descriptions may become brilliant at explanation. A mind trained on experience may become better at consequence. --- Full video from "Google DeepMind" and "Hannah Fry" YT channel (link in comment)

Rohan Paul

49,938 views • 1 month ago

Introducing Odyssey-2 Pro—a frontier world model that generates long-running, interactive simulations in 720p! We're also launching the first world model API, to enable devs to build magical apps. We're now in the GPT-2 era of world models. Let the explosion of apps commence!

Introducing Odyssey-2 Pro—a frontier world model that generates long-running, interactive simulations in 720p! We're also launching the first world model API, to enable devs to build magical apps. We're now in the GPT-2 era of world models. Let the explosion of apps commence!

Odyssey

166,093 views • 5 months ago

Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early results in autonomous general-purpose dexterous capabilities – fast, reactive, smooth, precise, bi-manual coordinated sensorimotor control.

Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early results in autonomous general-purpose dexterous capabilities – fast, reactive, smooth, precise, bi-manual coordinated sensorimotor control.

Generalist

277,934 views • 1 year ago

📁 Fei-Fei Li founder of World Labs, says the next leap in AI is not language. Human intelligence does not just speak, it moves, perceives, and acts in the physical world. Spatial intelligence is the real core of intelligence. From text to space, from models to 3D and 4D worlds, from understanding words to interacting with reality. The next chapter is not read, it is inhabited.

📁 Fei-Fei Li founder of World Labs, says the next leap in AI is not language. Human intelligence does not just speak, it moves, perceives, and acts in the physical world. Spatial intelligence is the real core of intelligence. From text to space, from models to 3D and 4D worlds, from understanding words to interacting with reality. The next chapter is not read, it is inhabited.

Jon Hernandez

20,592 views • 5 months ago