Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.

Sawyer Merritt

1,035,361 subscribers

254,296 views • 1 year ago •via X (Twitter)

Science & Technology News & Politics

Anya Rossi• Live Now

Private livecam show

11 Comments

Troy Teslike1 year ago

It feels like Nvidia is using Tesla as inspiration for their new products, which look pretty impressive.

Fast Company1 year ago

“Synthetic respondents are enabling us to create personas that simulate and replicate [human] behaviors with a high degree of fidelity,” says @NielsenIQ’s Ramon Melgarejo. And you can use this technology for your product innovation cycle too. #FCIF #ad

Sam Nissim1 year ago

First legit competitor to Tesla real world AI.

Tezlar1 year ago

Umm isn’t Tesla doing this as we speak with the 7 million robots they have roaming around the world?

WeirdNerd⚔️1 year ago

More confirmation Tesla is on the right track (and has been for years)

Jae Kim1 year ago

Yep. Seems Nvidia is coming after Tesla.

Bradford Ferguson1 year ago

NVIDIA has been unsuccessfully using synthetic data for many years now. It is a failed approach. Jensen didn't explain what is different now.

Kylead1 year ago

The whole presentation is mind-boggling.

YeahGloveSki1 year ago

Nvidia is Microsoft. Tesla is Apple.

Michael Tomlinson ✨1 year ago

It feels really odd to use an AI to generate photorealistic videos, only to use another AI to ‘learn’ about the world… from those photorealistic videos. (I mean I get why they’re doing it, but…) I wonder if we will soon be able to ’short circuit’ this process and just learn the distilled knowledge? Because this feels horrifically inefficient!

🇺🇸TRUTH SEEKER FOR THE PEOPLE 🇺🇸1 year ago

Nvidia's "Cosmos" isn't just a breakthrough; it's a leap into the future: Physical World Insight: Cosmos deciphers the complexities of our environment, opening new avenues for AI interaction with reality. Robotics Revolution: By generating synthetic data, Cosmos can teach robots to navigate and manipulate our world like never before, accelerating development beyond traditional constraints. This isn't just tech; it's the dawn of a new era where machines understand and interact with our world as naturally as we do.

Related Videos

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 views • 1 year ago

NVIDIA just introduced Cosmos, a platform for world foundation models designed for robotics. ⦿ It features advanced tokenizers, an AI-accelerated data pipeline, and integration with NVIDIA Omniverse. Humanoid makers 1X, Figure, and Agility are among the first to adopt Cosmos. ⦿ Cosmos generates synthetic, physics-based data, accelerating model training and customization. ⦿ It also features a CUDA-accelerated data processing pipeline that enables developers to process, curate, and label 20 million hours of videos in 14 days using the NVIDIA Blackwell platform.

NVIDIA just introduced Cosmos, a platform for world foundation models designed for robotics. ⦿ It features advanced tokenizers, an AI-accelerated data pipeline, and integration with NVIDIA Omniverse. Humanoid makers 1X, Figure, and Agility are among the first to adopt Cosmos. ⦿ Cosmos generates synthetic, physics-based data, accelerating model training and customization. ⦿ It also features a CUDA-accelerated data processing pipeline that enables developers to process, curate, and label 20 million hours of videos in 14 days using the NVIDIA Blackwell platform.

The Humanoid Hub

129,481 views • 1 year ago

🤔Want a principled way to RL your diffusion model? Check Data-regularized Reinforcement Learning (DDRL)! Post-train NVIDIA #Cosmos World Foundation models with a million GPU hours! 🤯 Novel formulation ➡️ Theoretically integrates SFT into RL ➡️ Robust to Reward Hacking 🛑 Details: #DDRL #Diffusion #RL #NVIDIA #Cosmos

🤔Want a principled way to RL your diffusion model? Check Data-regularized Reinforcement Learning (DDRL)! Post-train NVIDIA #Cosmos World Foundation models with a million GPU hours! 🤯 Novel formulation ➡️ Theoretically integrates SFT into RL ➡️ Robust to Reward Hacking 🛑 Details: #DDRL #Diffusion #RL #NVIDIA #Cosmos

Haotian Ye

78,082 views • 8 months ago

Introducing #NVIDIACosmos, the world foundation model platform built to advance physical #AI. Learn how, through integrations with NVIDIA Omniverse, developers can create physics-based, geospatially accurate scenarios. Watch the #CES2025 demo ➡️

Introducing #NVIDIACosmos, the world foundation model platform built to advance physical #AI. Learn how, through integrations with NVIDIA Omniverse, developers can create physics-based, geospatially accurate scenarios. Watch the #CES2025 demo ➡️

NVIDIA

142,176 views • 1 year ago

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

Max Zhaoshuo Li 李赵硕

1,078,418 views • 2 months ago

Very excited to share what our research team at Covariant has been working on: RFM-1, our latest Robotics Foundation Model. Built on top of high-quality multimodal data the Covariant robotics fleet has collected over the years, this project embodies our commitment to pushing the boundaries of how AI can interact with and understand the physical world. We're eager to share this milestone with the community and invite feedback and collaboration. Read more at

Very excited to share what our research team at Covariant has been working on: RFM-1, our latest Robotics Foundation Model. Built on top of high-quality multimodal data the Covariant robotics fleet has collected over the years, this project embodies our commitment to pushing the boundaries of how AI can interact with and understand the physical world. We're eager to share this milestone with the community and invite feedback and collaboration. Read more at

Rocky Duan

31,833 views • 2 years ago

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Grass

211,980 views • 2 years ago

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

Turing Post

13,836 views • 2 months ago

3rd Place: GrokWorld uses Grok Imagine as a world model to generate synthetic training data for robots — augmenting or replacing months of manual collection in hours. Aptura AI

3rd Place: GrokWorld uses Grok Imagine as a world model to generate synthetic training data for robots — augmenting or replacing months of manual collection in hours. Aptura AI

SpaceXAI

846,577 views • 7 months ago

We have nearly 40,000 hours of open robotics data on Check it out and train your own robotics policy or world model

We have nearly 40,000 hours of open robotics data on Check it out and train your own robotics policy or world model

Vai Viswanathan

32,053 views • 22 days ago

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

Roboflow

39,416 views • 2 months ago

The next frontier of AI is physical. 🤖 From simulation to synthetic data, developers are training robots to understand the real world with NVIDIA Cosmos, Isaac simulation frameworks and open-source physics engine, Newton. Watch the video to see how partners like Hon Hai Technology Group (Foxconn), Hexagon, Humanoid, Noble Machines, and Skild AI are building the future. 🎥

The next frontier of AI is physical. 🤖 From simulation to synthetic data, developers are training robots to understand the real world with NVIDIA Cosmos, Isaac simulation frameworks and open-source physics engine, Newton. Watch the video to see how partners like Hon Hai Technology Group (Foxconn), Hexagon, Humanoid, Noble Machines, and Skild AI are building the future. 🎥

NVIDIA Robotics

21,664 views • 4 months ago

"The “ChatGPT moment” for physical AI is nearly here. 👏 Cosmos open world foundation models understand reality to create and reason about rich synthetic worlds—so self-driving cars and robots can learn from endlessly varied, physics-aware scenarios. From a single image, 3D scene, or simulator trace, Cosmos generates realistic video, brings edge cases to life, and helps autonomous systems train for the long tail. Watch the #CES2026 demo now 🎥

"The “ChatGPT moment” for physical AI is nearly here. 👏 Cosmos open world foundation models understand reality to create and reason about rich synthetic worlds—so self-driving cars and robots can learn from endlessly varied, physics-aware scenarios. From a single image, 3D scene, or simulator trace, Cosmos generates realistic video, brings edge cases to life, and helps autonomous systems train for the long tail. Watch the #CES2026 demo now 🎥

NVIDIA AI

49,809 views • 7 months ago

Data for the Real World Diana & Austin Tindle AI has become superhuman at code, language, and images because it has enormous amounts of data to learn from. But our understanding of the physical world is still limited by sparse sensors and models built largely on intuition. Cheaper sensors and better foundation models could unlock entirely new datasets for energy, agriculture, logistics, construction, and the climate. Once you can accurately model the physical world, you can begin to control it.

Data for the Real World Diana & Austin Tindle AI has become superhuman at code, language, and images because it has enormous amounts of data to learn from. But our understanding of the physical world is still limited by sparse sensors and models built largely on intuition. Cheaper sensors and better foundation models could unlock entirely new datasets for energy, agriculture, logistics, construction, and the climate. Once you can accurately model the physical world, you can begin to control it.

Y Combinator

22,185 views • 27 days ago

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️

NVIDIA Robotics

14,344 views • 6 months ago

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

Ted Xiao

152,458 views • 1 year ago

Westlake Robotics just dropped the General Action Expert (GAE), a general large model that can generate arbitrary actions in real-time with very low latency. It allows the robot to become your physical avatar, executing any action like a shadow. #WestlakeRobotics #GAE #Robotics

RoboHub🤖

86,116 views • 10 months ago

Robotics has a massive, silent bottleneck. It isn’t just data collection—it’s the brutal 1x speed of the physical world. Genesis AI Genesis AI just unveiled Genesis World 1.0, and they are attempting to turn the notorious Sim2Real gap into a pure compute problem. Evaluating a robotics foundation model across edge cases usually means hundreds of hours of physical lab testing. With Genesis World 1.0, what traditionally takes nearly a week of continuous, real-world operation is being compressed into 30 minutes in simulation. What makes this different from just dropping a robot model into an off-the-shelf game engine? 1️⃣ Nyx Renderer: A custom, real-time path-traced engine rendering noise-free 1080p frames in under 4ms. Game engines use rasterization tricks that confuse AI; Nyx uses physically accurate multi-bounce lighting so the model's "eyes" see exactly what real sensors see. 2️⃣ Quadrants Compiler: A custom Python-to-GPU compiler to run heavily parallelized multi-physics simulations (rigid bodies, fluids, deformables) natively across architectures. 3️⃣ Evaluation First: They aren't rushing to train on synthetic data. They are using this purely for closed-loop evaluation to perfect the physics first, currently claiming an impressive 89% correlation with real-world hardware tests. If the industry can accurately evaluate models in simulation without the physical world bottleneck, humanoid development stops moving at wall-clock time and starts scaling with compute.

Robotics has a massive, silent bottleneck. It isn’t just data collection—it’s the brutal 1x speed of the physical world. Genesis AI Genesis AI just unveiled Genesis World 1.0, and they are attempting to turn the notorious Sim2Real gap into a pure compute problem. Evaluating a robotics foundation model across edge cases usually means hundreds of hours of physical lab testing. With Genesis World 1.0, what traditionally takes nearly a week of continuous, real-world operation is being compressed into 30 minutes in simulation. What makes this different from just dropping a robot model into an off-the-shelf game engine? 1️⃣ Nyx Renderer: A custom, real-time path-traced engine rendering noise-free 1080p frames in under 4ms. Game engines use rasterization tricks that confuse AI; Nyx uses physically accurate multi-bounce lighting so the model's "eyes" see exactly what real sensors see. 2️⃣ Quadrants Compiler: A custom Python-to-GPU compiler to run heavily parallelized multi-physics simulations (rigid bodies, fluids, deformables) natively across architectures. 3️⃣ Evaluation First: They aren't rushing to train on synthetic data. They are using this purely for closed-loop evaluation to perfect the physics first, currently claiming an impressive 89% correlation with real-world hardware tests. If the industry can accurately evaluate models in simulation without the physical world bottleneck, humanoid development stops moving at wall-clock time and starts scaling with compute.

Humanoids daily

17,240 views • 2 months ago