Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

At #GTC26 we are showing AlpaDreams — generative, interactive closed-loop simulation for autonomous driving. World models generate environments in real time, policies or drivers interact, and the simulator updates the world. If you are at GTC, come by the booth and try it out!

zianwang97's profile picture

Zian Wang

1,238 subscribers

16,972 Aufrufe • vor 5 Monaten •via X (Twitter)

#GTC26
LIVE

Anya Rossi• Live Now

Private livecam show

2 Kommentare

Profilbild von Zian Wang
Zian Wangvor 5 Monaten

Research blog:

Profilbild von Unseen Reality
Unseen Realityvor 5 Monaten

This is fantastic! Let's connect!

Ähnliche Videos

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.
1:31

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,676 Aufrufe • vor 3 Monaten

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation
2:52

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation

AGIBOT

28,219 Aufrufe • vor 5 Monaten

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️
0:33

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Google AI

102,711 Aufrufe • vor 1 Jahr

Roman Yampolskiy says we are probably in a computer simulation run by advanced AI to simulate an interesting time in history and we should try to hack out of it to take over avatar bodies in the "real world"
1:27

Roman Yampolskiy says we are probably in a computer simulation run by advanced AI to simulate an interesting time in history and we should try to hack out of it to take over avatar bodies in the "real world"

Tsarathustra

115,135 Aufrufe • vor 1 Jahr

The next frontier of autonomous driving is unlocked by reasoning models. NVIDIA Alpamayo brings together open AI models with reasoning capabilities, closed-loop simulation tools, and massive real-world driving datasets. Alpamayo 1 is a vision–language–action model that explains its own decisions through explicit reasoning traces, enabling trustworthy, humanlike decision-making. Together with NVIDIA’s Physical AI dataset and AlpaSim simulation, Alpamayo provides the tools and scale required to enable level 4 autonomous vehicles. ▶️ Watch now:
0:45

The next frontier of autonomous driving is unlocked by reasoning models. NVIDIA Alpamayo brings together open AI models with reasoning capabilities, closed-loop simulation tools, and massive real-world driving datasets. Alpamayo 1 is a vision–language–action model that explains its own decisions through explicit reasoning traces, enabling trustworthy, humanlike decision-making. Together with NVIDIA’s Physical AI dataset and AlpaSim simulation, Alpamayo provides the tools and scale required to enable level 4 autonomous vehicles. ▶️ Watch now:

NVIDIA DRIVE

35,324 Aufrufe • vor 8 Monaten

Training fully autonomous robots takes more data than the real world can easily provide. Simulation lets us train, test, and deploy autonomous software without being limited by robots, space, or cost. Here’s OM1 running in simulation side-by-side to the real world.
1:26

Training fully autonomous robots takes more data than the real world can easily provide. Simulation lets us train, test, and deploy autonomous software without being limited by robots, space, or cost. Here’s OM1 running in simulation side-by-side to the real world.

OpenMind

121,315 Aufrufe • vor 9 Monaten

XPENG is thrilled to introduce X-World, our controllable multi-view generative world model powered by video diffusion technology. Pushing the boundaries of smart mobility, X-World delivers real-time response and continuous multi-view generation to simulate complex driving environments. Xianming Liu, Head of XPENG’s General AI Center, emphasizes: "Beyond generation, X-World serves as the scalable infrastructure critical for training and validating the next leap in autonomous driving." $XPEV
1:36

XPENG is thrilled to introduce X-World, our controllable multi-view generative world model powered by video diffusion technology. Pushing the boundaries of smart mobility, X-World delivers real-time response and continuous multi-view generation to simulate complex driving environments. Xianming Liu, Head of XPENG’s General AI Center, emphasizes: "Beyond generation, X-World serves as the scalable infrastructure critical for training and validating the next leap in autonomous driving." $XPEV

XPENG

4,070,965 Aufrufe • vor 5 Monaten

we are introducing a state-of-the-art real-time object detection model, RF-DETR RF-DETR outperforms all existing object detection models on real world datasets and is the first real-time model to achieve 60+ Average Precision on COCO talked w/ NVIDIA about it at GTC:
2:31

we are introducing a state-of-the-art real-time object detection model, RF-DETR RF-DETR outperforms all existing object detection models on real world datasets and is the first real-time model to achieve 60+ Average Precision on COCO talked w/ NVIDIA about it at GTC:

Joseph Nelson

18,231 Aufrufe • vor 1 Jahr

this is the craziest AI we have now Google’s Genie 3 can generate any interactive 3D world, you can literally walk around in it and interact with things.. example 1: step inside an old painting or a real location and explore it…
0:26

this is the craziest AI we have now Google’s Genie 3 can generate any interactive 3D world, you can literally walk around in it and interact with things.. example 1: step inside an old painting or a real location and explore it…

el.cine

238,117 Aufrufe • vor 1 Jahr

Step into the map with the Street View grounding feature in Project Genie from Google DeepMind and Google Labs. Announced at I/O, this research prototype uses locations from Google Maps Street View as a foundation, letting you generate and explore interactive, 360-degree virtual environments from just a text prompt or real-world starting place. Cool, right? But… How does it actually work? 🤔 As an experimental tool, Project Genie tackles the "blank space" problem (showing both what’s in front of the camera and behind it) by utilizing Street View data to realistically generate a 360-degree view of the location you selected as the starting point for your world to generate from. Worlds generated by Genie are far more dynamic and rich because they’re created frame-by-frame based on the world description and user actions. By predicting each subsequent frame, Genie is able to simulate what it looks like to swim across an ocean, or hike to the top of a peak, marking a massive shift in interactive media and simulation pipelines. What real-world place would you want to step into and explore?
0:27

Step into the map with the Street View grounding feature in Project Genie from Google DeepMind and Google Labs. Announced at I/O, this research prototype uses locations from Google Maps Street View as a foundation, letting you generate and explore interactive, 360-degree virtual environments from just a text prompt or real-world starting place. Cool, right? But… How does it actually work? 🤔 As an experimental tool, Project Genie tackles the "blank space" problem (showing both what’s in front of the camera and behind it) by utilizing Street View data to realistically generate a 360-degree view of the location you selected as the starting point for your world to generate from. Worlds generated by Genie are far more dynamic and rich because they’re created frame-by-frame based on the world description and user actions. By predicting each subsequent frame, Genie is able to simulate what it looks like to swim across an ocean, or hike to the top of a peak, marking a massive shift in interactive media and simulation pipelines. What real-world place would you want to step into and explore?

Google AI

4,043,116 Aufrufe • vor 2 Monaten

"Data stops being something you collect. It becomes something you compute." At #SIGGRAPH2026, Ming-Yu Liu, VP of NVIDIA's Cosmos Lab, laid out why world models are the data engine of physical AI. Plus, he announced Cosmos-Dreams, a neural closed-loop simulator, and showed off a full demo of it in action. You can watch the full keynote here:
1:16

"Data stops being something you collect. It becomes something you compute." At #SIGGRAPH2026, Ming-Yu Liu, VP of NVIDIA's Cosmos Lab, laid out why world models are the data engine of physical AI. Plus, he announced Cosmos-Dreams, a neural closed-loop simulator, and showed off a full demo of it in action. You can watch the full keynote here:

NVIDIA AI

15,771 Aufrufe • vor 1 Monat

Rollouts in the real world are slow and expensive. What if we could rollout trajectories entirely inside a world model (WM)? Introducing 🚀Ctrl-World🚀, a generative manipulation WM that can interact with advanced VLA policy in imagination. 🧵1/6
1:37

Rollouts in the real world are slow and expensive. What if we could rollout trajectories entirely inside a world model (WM)? Introducing 🚀Ctrl-World🚀, a generative manipulation WM that can interact with advanced VLA policy in imagination. 🧵1/6

Yanjiang Guo

95,816 Aufrufe • vor 10 Monaten

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.
1:29

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

17,065 Aufrufe • vor 7 Monaten

How good are your skill checks? Come test them out at the Dandy’s World booth at LVL Up Expo! And yes, the Twisteds noticing sound is scarier in IRL.
0:30

How good are your skill checks? Come test them out at the Dandy’s World booth at LVL Up Expo! And yes, the Twisteds noticing sound is scarier in IRL.

CultureFly

4,970,322 Aufrufe • vor 4 Monaten

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado
1:30

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

a16z

35,733 Aufrufe • vor 1 Monat

Genie 3 feels like a watershed moment for world models 🌐: we can now generate multi-minute, real-time interactive simulations of any imaginable world. This could be the key missing piece for embodied AGI… and it can also create beautiful beaches with my dog, playable real time
0:30

Genie 3 feels like a watershed moment for world models 🌐: we can now generate multi-minute, real-time interactive simulations of any imaginable world. This could be the key missing piece for embodied AGI… and it can also create beautiful beaches with my dog, playable real time

Jack Parker-Holder

2,083,836 Aufrufe • vor 1 Jahr

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n
3:26

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,821,069 Aufrufe • vor 1 Jahr

2GOOD4U out now in the Philippines and for the rest of the world... dropping at midnight in your time zone, wherever you are ♡
0:39

2GOOD4U out now in the Philippines and for the rest of the world... dropping at midnight in your time zone, wherever you are ♡

denise julia ♡

36,528 Aufrufe • vor 5 Monaten

Introducing Driver — a transformer-based, real-time, vision-only path prediction model for highway and urban driving. It predicts the vehicle’s future path using only camera-based perception—no Lidar, HD maps, or additional sensors required. We deployed it in closed-loop simulation with CARLA Simulator + GenSim-2, our generative AI model that re-renders realistic camera visuals. (top: original CARLA; bottom: GenSim-2 re-render). See more details in thread👇
0:48

Introducing Driver — a transformer-based, real-time, vision-only path prediction model for highway and urban driving. It predicts the vehicle’s future path using only camera-based perception—no Lidar, HD maps, or additional sensors required. We deployed it in closed-loop simulation with CARLA Simulator + GenSim-2, our generative AI model that re-renders realistic camera visuals. (top: original CARLA; bottom: GenSim-2 re-render). See more details in thread👇

Helm.ai

4,040,913 Aufrufe • vor 1 Jahr

Real-time World Models are the next AI frontier. Today, we reactor are taking the first step towards this reality: our early preview lets you experience worlds generated in real-time, running on our global low-latency infrastructure. Try it now:
0:34

Real-time World Models are the next AI frontier. Today, we reactor are taking the first step towards this reality: our early preview lets you experience worlds generated in real-time, running on our global low-latency infrastructure. Try it now:

Bryce Schmidtchen

8,491,427 Aufrufe • vor 4 Monaten