Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail...

TheHumanoidHub's profile picture

The Humanoid Hub

112,666 subscribers

17,065 Aufrufe • vor 9 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐
3:04

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐

Yunzhu Li

39,960 Aufrufe • vor 1 Jahr

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation
2:52

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation

AGIBOT

28,219 Aufrufe • vor 7 Monaten

How do you grade a world model? Against what the robot really did. Picture a robot gripper closing on a piece of fruit. In the real world, it fails to grasp it. Now hand the same first frame and the same action sequence to an action-conditioned world model (ACWM). The rollout shows a tidy, successful pick. Nothing on screen looks wrong. That is the moment WorldOracle Bench was built for. Created by AGIBOT and NVIDIA together with Princeton, UT Austin and the University of Toronto, it is an evaluation platform for robot behavior models. It replays the real, recorded actions through a model and checks the predicted future against the paired real execution. The stakes are practical. The paper points to data generation, scalable policy evaluation and closed-loop policy learning as potential uses of ACWMs. If a model gets the physical consequences wrong, everything built on top of it inherits the error. The authors call the property they are testing execution coherence: does the model follow the actions it is given, even the bad ones, across different embodiments and action sources, and over long rollouts? What makes the setup unusual is where the evidence comes from. The paper compares eight existing embodied world-model benchmarks, and none of them combines action-conditioned evaluation, real-world data, failed executions, long episodes, multiple embodiments, multi-view outputs and policy-generated actions. WorldOracle Bench does. Its evaluation set has 335 episodes: 100 on AGIBOT G2, 75 on YAM and 100 with a human hand, all human-operated, plus 60 rollouts where a learned policy drove a real AGIBOT G2 robot. About 40% of the human-operated segments are failures, so a model cannot get by on success stories.
0:29

How do you grade a world model? Against what the robot really did. Picture a robot gripper closing on a piece of fruit. In the real world, it fails to grasp it. Now hand the same first frame and the same action sequence to an action-conditioned world model (ACWM). The rollout shows a tidy, successful pick. Nothing on screen looks wrong. That is the moment WorldOracle Bench was built for. Created by AGIBOT and NVIDIA together with Princeton, UT Austin and the University of Toronto, it is an evaluation platform for robot behavior models. It replays the real, recorded actions through a model and checks the predicted future against the paired real execution. The stakes are practical. The paper points to data generation, scalable policy evaluation and closed-loop policy learning as potential uses of ACWMs. If a model gets the physical consequences wrong, everything built on top of it inherits the error. The authors call the property they are testing execution coherence: does the model follow the actions it is given, even the bad ones, across different embodiments and action sources, and over long rollouts? What makes the setup unusual is where the evidence comes from. The paper compares eight existing embodied world-model benchmarks, and none of them combines action-conditioned evaluation, real-world data, failed executions, long episodes, multiple embodiments, multi-view outputs and policy-generated actions. WorldOracle Bench does. Its evaluation set has 335 episodes: 100 on AGIBOT G2, 75 on YAM and 100 with a human hand, all human-operated, plus 60 rollouts where a learned policy drove a real AGIBOT G2 robot. About 40% of the human-operated segments are failures, so a model cannot get by on success stories.

Science girl

82,284 Aufrufe • vor 11 Tagen

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.
1:31

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,693 Aufrufe • vor 5 Monaten

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.
1:03

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.

Sawyer Merritt

133,382 Aufrufe • vor 9 Monaten

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.
0:40

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.

Runway

13,464 Aufrufe • vor 8 Monaten

Odyssey launches Odyssey-3, a foundation world model that sets a new state of the art on Physics-IQ Verified (66.1) for video-to-video physical simulation. The same pretrained model generates real-time interactive environments from prompts and adapts to control robots, humanoids, cars, and drones with limited task-specific data.
0:52

Odyssey launches Odyssey-3, a foundation world model that sets a new state of the art on Physics-IQ Verified (66.1) for video-to-video physical simulation. The same pretrained model generates real-time interactive environments from prompts and adapts to control robots, humanoids, cars, and drones with limited task-specific data.

Documenting AGI

38,907 Aufrufe • vor 1 Monat

Robotics has a massive, silent bottleneck. It isn’t just data collection—it’s the brutal 1x speed of the physical world. Genesis AI Genesis AI just unveiled Genesis World 1.0, and they are attempting to turn the notorious Sim2Real gap into a pure compute problem. Evaluating a robotics foundation model across edge cases usually means hundreds of hours of physical lab testing. With Genesis World 1.0, what traditionally takes nearly a week of continuous, real-world operation is being compressed into 30 minutes in simulation. What makes this different from just dropping a robot model into an off-the-shelf game engine? 1️⃣ Nyx Renderer: A custom, real-time path-traced engine rendering noise-free 1080p frames in under 4ms. Game engines use rasterization tricks that confuse AI; Nyx uses physically accurate multi-bounce lighting so the model's "eyes" see exactly what real sensors see. 2️⃣ Quadrants Compiler: A custom Python-to-GPU compiler to run heavily parallelized multi-physics simulations (rigid bodies, fluids, deformables) natively across architectures. 3️⃣ Evaluation First: They aren't rushing to train on synthetic data. They are using this purely for closed-loop evaluation to perfect the physics first, currently claiming an impressive 89% correlation with real-world hardware tests. If the industry can accurately evaluate models in simulation without the physical world bottleneck, humanoid development stops moving at wall-clock time and starts scaling with compute.
1:36

Robotics has a massive, silent bottleneck. It isn’t just data collection—it’s the brutal 1x speed of the physical world. Genesis AI Genesis AI just unveiled Genesis World 1.0, and they are attempting to turn the notorious Sim2Real gap into a pure compute problem. Evaluating a robotics foundation model across edge cases usually means hundreds of hours of physical lab testing. With Genesis World 1.0, what traditionally takes nearly a week of continuous, real-world operation is being compressed into 30 minutes in simulation. What makes this different from just dropping a robot model into an off-the-shelf game engine? 1️⃣ Nyx Renderer: A custom, real-time path-traced engine rendering noise-free 1080p frames in under 4ms. Game engines use rasterization tricks that confuse AI; Nyx uses physically accurate multi-bounce lighting so the model's "eyes" see exactly what real sensors see. 2️⃣ Quadrants Compiler: A custom Python-to-GPU compiler to run heavily parallelized multi-physics simulations (rigid bodies, fluids, deformables) natively across architectures. 3️⃣ Evaluation First: They aren't rushing to train on synthetic data. They are using this purely for closed-loop evaluation to perfect the physics first, currently claiming an impressive 89% correlation with real-world hardware tests. If the industry can accurately evaluate models in simulation without the physical world bottleneck, humanoid development stops moving at wall-clock time and starts scaling with compute.

Humanoids daily

17,302 Aufrufe • vor 5 Monaten

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:
0:41

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

14,222 Aufrufe • vor 5 Monaten

Self-learning worked in AlphaGo because the world was a clean rulebook, says Demis Hassabis. Real life has no perfect rulebook, so long-term planning needs a strong “world model” to simulate what happens next. Without that simulation, a robot has to guess in the real world—where mistakes can be costly and irreversible. Better world models are the first missing piece, says Demis Hassabis.
1:38

Self-learning worked in AlphaGo because the world was a clean rulebook, says Demis Hassabis. Real life has no perfect rulebook, so long-term planning needs a strong “world model” to simulate what happens next. Without that simulation, a robot has to guess in the real world—where mistakes can be costly and irreversible. Better world models are the first missing piece, says Demis Hassabis.

Wes Roth

15,757 Aufrufe • vor 9 Monaten

The next frontier of autonomous driving is unlocked by reasoning models. NVIDIA Alpamayo brings together open AI models with reasoning capabilities, closed-loop simulation tools, and massive real-world driving datasets. Alpamayo 1 is a vision–language–action model that explains its own decisions through explicit reasoning traces, enabling trustworthy, humanlike decision-making. Together with NVIDIA’s Physical AI dataset and AlpaSim simulation, Alpamayo provides the tools and scale required to enable level 4 autonomous vehicles. ▶️ Watch now:
0:45

The next frontier of autonomous driving is unlocked by reasoning models. NVIDIA Alpamayo brings together open AI models with reasoning capabilities, closed-loop simulation tools, and massive real-world driving datasets. Alpamayo 1 is a vision–language–action model that explains its own decisions through explicit reasoning traces, enabling trustworthy, humanlike decision-making. Together with NVIDIA’s Physical AI dataset and AlpaSim simulation, Alpamayo provides the tools and scale required to enable level 4 autonomous vehicles. ▶️ Watch now:

NVIDIA DRIVE

35,324 Aufrufe • vor 10 Monaten

I texted someone inside a simulation. She had to go upstairs and find her phone before she could text me back. The video model didn’t decide what happened. This is Argos, my stateful simulation engine, rendered through H3 using Fal Director. Thanks to fal for the generous credits helping me bring it to life. Underneath the video is a persistent world: characters, objects, locations, and systems. The characters have their own cognition, memories, and goals. They choose actions; the simulation determines what actually happens. Her phone has a location. Messages have delivery and read states. Typing a reply takes time. If I move the phone before she gets there, it’s gone—even if she remembers leaving it there. Argos turns those changing world states into instructions for H3 to depict. You can also see her character thought stream. The video currently trails the simulation a little; I’m working on aligning the two. Next: connect this to SMS, so you can text someone inside the world and receive their reply. Or whisper an impulse into her thoughts and see whether she follows it. The bigger ambition is a world that keeps growing while its characters inhabit it—with agents writing and maintaining the simulation’s systems. I’ll be building more of this in public. Not just a video that looks like a world. A world that keeps existing underneath the video.
1:42

I texted someone inside a simulation. She had to go upstairs and find her phone before she could text me back. The video model didn’t decide what happened. This is Argos, my stateful simulation engine, rendered through H3 using Fal Director. Thanks to fal for the generous credits helping me bring it to life. Underneath the video is a persistent world: characters, objects, locations, and systems. The characters have their own cognition, memories, and goals. They choose actions; the simulation determines what actually happens. Her phone has a location. Messages have delivery and read states. Typing a reply takes time. If I move the phone before she gets there, it’s gone—even if she remembers leaving it there. Argos turns those changing world states into instructions for H3 to depict. You can also see her character thought stream. The video currently trails the simulation a little; I’m working on aligning the two. Next: connect this to SMS, so you can text someone inside the world and receive their reply. Or whisper an impulse into her thoughts and see whether she follows it. The bigger ambition is a world that keeps growing while its characters inhabit it—with agents writing and maintaining the simulation’s systems. I’ll be building more of this in public. Not just a video that looks like a world. A world that keeps existing underneath the video.

Parzival - ∞/89

495,635 Aufrufe • vor 2 Monaten

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.
1:01

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

Praveen Kumar Verma

138,017 Aufrufe • vor 2 Monaten

In the era of 10T-parameter foundation models 😈, can we still build next-generation world representations using 🦥 a single RTX 3090 GPU? We introduce WorldString (a name inspired by string theory), a tiny neural network representation for reconstructing dynamic world instances, ranging from articulated motion and skeletal skinning to soft-body deformation. 🔥 We view object representation as the core primitive of world modeling: the basic executable unit from which perception, interaction, simulation, and generation can be built. With a compact neural basis, WorldString aims to connect dynamic object modeling with neural simulation and foundation models, enabling a scalable path toward richer representations of the physical world.
0:29

In the era of 10T-parameter foundation models 😈, can we still build next-generation world representations using 🦥 a single RTX 3090 GPU? We introduce WorldString (a name inspired by string theory), a tiny neural network representation for reconstructing dynamic world instances, ranging from articulated motion and skeletal skinning to soft-body deformation. 🔥 We view object representation as the core primitive of world modeling: the basic executable unit from which perception, interaction, simulation, and generation can be built. With a compact neural basis, WorldString aims to connect dynamic object modeling with neural simulation and foundation models, enabling a scalable path toward richer representations of the physical world.

Xueyan Zou@CoRL26

39,574 Aufrufe • vor 4 Monaten

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 8 Monaten

It’s easy to watch this video and see a pre-recorded clip, but what you’re really watching is a frame-by-frame, interactive simulation of the world. Odyssey-2 Pro—a general-purpose world model—is prompted and then rolls out a minutes-long, open-ended simulation.
1:41

It’s easy to watch this video and see a pre-recorded clip, but what you’re really watching is a frame-by-frame, interactive simulation of the world. Odyssey-2 Pro—a general-purpose world model—is prompted and then rolls out a minutes-long, open-ended simulation.

Odyssey

32,529 Aufrufe • vor 10 Monaten

Today at Stanford, Fei-Fei Li (Fei-Fei Li),Cofounder/CEO World Labs, gave one of the clearest explanations I’ve heard of what a World Model really is. She broke it down into three layers: 1️⃣ Rendering — What does the world look like? This is where most of today’s video generation models operate: generating increasingly realistic and beautiful pixels. The question is: Can AI generate what the world looks like? The primary consumer is humans. 2️⃣ Simulation — How does the world actually work? Fei-Fei gave a simple example: “How will this bottle move? If I pour the water out, how will the water flow?” This goes far beyond generating something that looks realistic. The model needs to understand physics, spatial relationships, cause and effect, and how the world changes over time. The consumers are both humans and machines. 3️⃣ Planning — What should happen next? This is where things get really interesting. AI doesn't just render the world or simulate what might happen. It uses its understanding of the world to decide: What should I do next? At this layer, the primary consumer is the machine itself. And this connects directly to two enormous opportunities: Autonomous driving and robotics. The progression is powerful: Rendering → Simulation → Planning The real promise of World Models isn't simply generating better videos. It's building AI that can understand the world, predict what happens next, and ultimately take intelligent action in the physical world.
3:54

Today at Stanford, Fei-Fei Li (Fei-Fei Li),Cofounder/CEO World Labs, gave one of the clearest explanations I’ve heard of what a World Model really is. She broke it down into three layers: 1️⃣ Rendering — What does the world look like? This is where most of today’s video generation models operate: generating increasingly realistic and beautiful pixels. The question is: Can AI generate what the world looks like? The primary consumer is humans. 2️⃣ Simulation — How does the world actually work? Fei-Fei gave a simple example: “How will this bottle move? If I pour the water out, how will the water flow?” This goes far beyond generating something that looks realistic. The model needs to understand physics, spatial relationships, cause and effect, and how the world changes over time. The consumers are both humans and machines. 3️⃣ Planning — What should happen next? This is where things get really interesting. AI doesn't just render the world or simulate what might happen. It uses its understanding of the world to decide: What should I do next? At this layer, the primary consumer is the machine itself. And this connects directly to two enormous opportunities: Autonomous driving and robotics. The progression is powerful: Rendering → Simulation → Planning The real promise of World Models isn't simply generating better videos. It's building AI that can understand the world, predict what happens next, and ultimately take intelligent action in the physical world.

PaulFang

12,066 Aufrufe • vor 3 Monaten

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.
1:52

The "reality gap" is narrowing. Sanctuary AI has demonstrated a sim-to-real breakthrough: "zero-shot" transfer for complex in-hand manipulation. By leveraging high-DOF hydraulic hands, the company successfully reoriented a cube in the physical world using a policy trained entirely in simulation.

Humanoids daily

12,391 Aufrufe • vor 7 Monaten

$AMD is building its answer to $NVDA Cosmos by owning World Labs and moving into world models for robotics, autonomy and 3D simulation. The model layer often decides the hardware layer so AMD now gets a shot at winning physical AI developers earlier.
1:53

$AMD is building its answer to $NVDA Cosmos by owning World Labs and moving into world models for robotics, autonomy and 3D simulation. The model layer often decides the hardware layer so AMD now gets a shot at winning physical AI developers earlier.

Shay Boloor

71,595 Aufrufe • vor 1 Monat

NVIDIA CEO Jensen Huang on Tesla FSD at CES: “Tesla’s FSD stack is completely world-class. They’ve been working on it for quite some time. It’s world-class. Not only in the number of miles it has, but it’s world-class in the way that it’s designed, the way they do training to data collection and curation, synthetic data generation, all of their simulation technologies. Of course, the latest generation is end-to-end full self-driving, meaning it’s just one large model that is end-to-end trained. Elon’s AV system is, in every way, 100% state-of-the-art. And so I’m really quite impressed by the technology. I have it and I drive it in our house and it works incredibly well.”
1:11

NVIDIA CEO Jensen Huang on Tesla FSD at CES: “Tesla’s FSD stack is completely world-class. They’ve been working on it for quite some time. It’s world-class. Not only in the number of miles it has, but it’s world-class in the way that it’s designed, the way they do training to data collection and curation, synthetic data generation, all of their simulation technologies. Of course, the latest generation is end-to-end full self-driving, meaning it’s just one large model that is end-to-end trained. Elon’s AV system is, in every way, 100% state-of-the-art. And so I’m really quite impressed by the technology. I have it and I drive it in our house and it works incredibly well.”

Nic Cruz Patane

52,826 Aufrufe • vor 10 Monaten