Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

1/ NitroGen: NVIDIA's new image-to-action model! NitroGen, a vision-action foundation model for generalist gaming agents that is trained on 40,000 hours of gameplay videos across more than 1,000 games. Gaming is a significant factor in AI training. Google DeepMind trained AI early on Starcraft 2, and OpenAI on Dota... show more

Chubby♨️

96,423 subscribers

55,280 views • 7 months ago •via X (Twitter)

Gaming Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Google DeepMind just dropped 'Genie', an AI that can generate interactive video games. This is a huge deal. Genie is trained on 200,000 hours of unsupervised public internet gaming videos and can generate video games from a single prompt or image. But here's what's insane: Despite not being trained on action or text annotations, the foundation model can determine who the main character is and enable a user to control that character in the generated world. It does this through its Latent Action Model, Video Tokenizer, and Dynamics Model (will go more in-depth on this in tomorrow's newsletter for those interested). And for those asking, yes, it's research-only and not publicly available (here come the Google memes), and it does come with some limitations, like only currently creating games at 1FPS. But this is the worst AI will ever be. Anyone will be able to create their own entirely imagined virtual worlds soon, and that's a wild sentence to say out loud.

Google DeepMind just dropped 'Genie', an AI that can generate interactive video games. This is a huge deal. Genie is trained on 200,000 hours of unsupervised public internet gaming videos and can generate video games from a single prompt or image. But here's what's insane: Despite not being trained on action or text annotations, the foundation model can determine who the main character is and enable a user to control that character in the generated world. It does this through its Latent Action Model, Video Tokenizer, and Dynamics Model (will go more in-depth on this in tomorrow's newsletter for those interested). And for those asking, yes, it's research-only and not publicly available (here come the Google memes), and it does come with some limitations, like only currently creating games at 1FPS. But this is the worst AI will ever be. Anyone will be able to create their own entirely imagined virtual worlds soon, and that's a wild sentence to say out loud.

Rowan Cheung

3,377,465 views • 2 years ago

NVIDIA just revealed MotionBricks, a new AI model trained on more than 350,000 motion clips. What’s really impressive is that the same model can animate a digital character and control a humanoid robot in the real world.

NVIDIA just revealed MotionBricks, a new AI model trained on more than 350,000 motion clips. What’s really impressive is that the same model can animate a digital character and control a humanoid robot in the real world.

Justin Ryan

84,887 views • 28 days ago

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,882 views • 1 month ago

Chinese robotics company Astribot released their latest World-Action Model (WAM), Lumo-2. Technical breakdown: - based on a frozen 🥶 Qwen-3.5 4B VLM - trained in 3 progressive stages: 1. Action is aligned with latent world dynamics (an abstract representation of action). Real-world actions are anchored to physical constraints, while the latent space is guided to focus on motion-relevant changes. This bidirectional relationship makes the model physically grounded -> critical for a world model. 2. Action is aligned with vision and language. Reusing the vision backbone and action encoder from the frozen VLM, the authors add a custom vocabulary (for new actions), a semantic module, an action decoder, and an action projector. This aligns the (new) action representations with the (existing) vision-language semantic space. Most importantly: it builds a direct mapping from natural-language instructions to motor execution. 3. End-to-end training on language, video, and robot data. Only the new modules (everything outside the frozen backbone) are trained end-to-end across temporal reasoning, physical understanding, long-horizon, and dexterous manipulation. At the end of the day, Lumo-2 is not the best on benchmarks, but that's not the point. What's genuinely new: - a way to combine latent world modeling and action generation through progressive alignment - a physically-grounded latent dynamics space - it lifts performance on unseen objects using un-annotated human egocentric video + Vision Pro captures, no special transfer algorithm needed Why it matters: - the whole model is thin trainable adapters (semantic module, action decoder/projector) on a frozen 4B backbone (cheap) - that scale is suited for real-time embedded inference (~2.71× decode speedup, no accuracy loss) - its real moat is long-horizon execution, where the added temporal memory pays off far more than on any other task As a result, this robot can now make your latte (5x sped up video):

Chinese robotics company Astribot released their latest World-Action Model (WAM), Lumo-2. Technical breakdown: - based on a frozen 🥶 Qwen-3.5 4B VLM - trained in 3 progressive stages: 1. Action is aligned with latent world dynamics (an abstract representation of action). Real-world actions are anchored to physical constraints, while the latent space is guided to focus on motion-relevant changes. This bidirectional relationship makes the model physically grounded -> critical for a world model. 2. Action is aligned with vision and language. Reusing the vision backbone and action encoder from the frozen VLM, the authors add a custom vocabulary (for new actions), a semantic module, an action decoder, and an action projector. This aligns the (new) action representations with the (existing) vision-language semantic space. Most importantly: it builds a direct mapping from natural-language instructions to motor execution. 3. End-to-end training on language, video, and robot data. Only the new modules (everything outside the frozen backbone) are trained end-to-end across temporal reasoning, physical understanding, long-horizon, and dexterous manipulation. At the end of the day, Lumo-2 is not the best on benchmarks, but that's not the point. What's genuinely new: - a way to combine latent world modeling and action generation through progressive alignment - a physically-grounded latent dynamics space - it lifts performance on unseen objects using un-annotated human egocentric video + Vision Pro captures, no special transfer algorithm needed Why it matters: - the whole model is thin trainable adapters (semantic module, action decoder/projector) on a frozen 4B backbone (cheap) - that scale is suited for real-time embedded inference (~2.71× decode speedup, no accuracy loss) - its real moat is long-horizon execution, where the added temporal memory pays off far more than on any other task As a result, this robot can now make your latte (5x sped up video):

Léo

32,173 views • 25 days ago

Google DeepMind CEO Shows How Their New AI Model Has Reverse Engineered Physics “This model is reverse engineering intuitive physics. It's watched many millions of videos … and just from that it's kind of reverse engineered how a lot of the world works.”

Google DeepMind CEO Shows How Their New AI Model Has Reverse Engineered Physics “This model is reverse engineering intuitive physics. It's watched many millions of videos … and just from that it's kind of reverse engineered how a lot of the world works.”

Chief Nerd

34,875 views • 11 months ago

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

AK

684,372 views • 2 years ago

Disney just revealed a self-walking Olaf robot. It's trained on 10,000+ hours of AI motion simulations built with NVIDIA and DeepMind. It even does Olaf’s signature waddling walk from the films. Here’s why this matters far beyond theme parks:

Disney just revealed a self-walking Olaf robot. It's trained on 10,000+ hours of AI motion simulations built with NVIDIA and DeepMind. It even does Olaf’s signature waddling walk from the films. Here’s why this matters far beyond theme parks:

Elad Inbar

859,983 views • 8 months ago

$American doctor put this picture of a very obviously fractured bone into Claude AI and said “Evaluate this X-Ray” The results should horrify you This is why people should not be trusting anything medical to AI, unless they AI has been trained for medical The obvious pushback from people is this AI model probably wasn’t trained for this, well then why is it giving an answer?? It should prompt you that it isn’t trained on this data and it cannot evaluate this$

American doctor put this picture of a very obviously fractured bone into Claude AI and said “Evaluate this X-Ray” The results should horrify you This is why people should not be trusting anything medical to AI, unless they AI has been trained for medical The obvious pushback from people is this AI model probably wasn’t trained for this, well then why is it giving an answer?? It should prompt you that it isn’t trained on this data and it cannot evaluate this

Wall Street Apes

201,685 views • 1 month ago

Big News! Meta just released Segment Anything, a new AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.

Big News! Meta just released Segment Anything, a new AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.

Lior Alexander

290,330 views • 3 years ago

New incredible AI image generator is here NEX AI just dropped Ikon 2 , a new foundation model that beat Deepseek in accuracy with way fewer GPU hours. Ikon's image generator bring realism to next level here's why it's mind blowing:

New incredible AI image generator is here NEX AI just dropped Ikon 2 , a new foundation model that beat Deepseek in accuracy with way fewer GPU hours. Ikon's image generator bring realism to next level here's why it's mind blowing:

el.cine

99,907 views • 1 year ago

Nvidia is developing a new AI chip for China based on its latest Blackwell architecture that will be more powerful than the H20 model it is currently allowed to sell there

Nvidia is developing a new AI chip for China based on its latest Blackwell architecture that will be more powerful than the H20 model it is currently allowed to sell there

Reuters

43,599 views • 1 year ago

Developers are coming to Avalanche for AI; it offers customizable compute and data structures, plus a robust ecosystem across finance, gaming, payments, and more. Listen to KITE AI talk about how this synergy is powering new AI agents and models for every sector. For more: 👇

Developers are coming to Avalanche for AI; it offers customizable compute and data structures, plus a robust ecosystem across finance, gaming, payments, and more. Listen to KITE AI talk about how this synergy is powering new AI agents and models for every sector. For more: 👇

Avalanche🔺

22,981 views • 1 year ago

We have isolated the vocals of over 28,000 songs: Not for any music AI model at all. Not to clone these vocals or the singer’s style. Not anything musical, although I have made some interesting discoveries. It is a new AI model that is being trained on a new paradigm.

We have isolated the vocals of over 28,000 songs: Not for any music AI model at all. Not to clone these vocals or the singer’s style. Not anything musical, although I have made some interesting discoveries. It is a new AI model that is being trained on a new paradigm.

Brian Roemmele

56,584 views • 4 months ago

LingBot-VLA 2.0 is an impressive new embodied model. Open source and is trained across diverse robot configurations, from single-arm robots to humanoid platforms. It packs 60K hours of curated robot and human data into one generalist policy. It improves robots on difficult long-horizon tasks. Great release by Robbyant.

LingBot-VLA 2.0 is an impressive new embodied model. Open source and is trained across diverse robot configurations, from single-arm robots to humanoid platforms. It packs 60K hours of curated robot and human data into one generalist policy. It improves robots on difficult long-horizon tasks. Great release by Robbyant.

elvis

10,464 views • 1 month ago

Today we’re announcing X-Cell — Xaira’s first step toward a virtual cell. 🧬 A foundation model that predicts how gene expression changes under causal perturbations — across cell types, conditions, and even unseen biology. This is not trained on observational atlases. It is trained on interventions. 🧵👇

Today we’re announcing X-Cell — Xaira’s first step toward a virtual cell. 🧬 A foundation model that predicts how gene expression changes under causal perturbations — across cell types, conditions, and even unseen biology. This is not trained on observational atlases. It is trained on interventions. 🧵👇

Bo Wang

174,098 views • 5 months ago

GPT Image 2 is UNLIMITED FREE for new AITOPIA customers. For today’s launch only, customers who join and subscribe to a plan within the next 24 hours get GPT Image 2 UNLIMITED FREE in their first 4 hours. Access latest AI models and agents in one subscription on AITOPIA. Trusted by more than 3 million users, together with 150+ AI Agents, 500+ AI models. Subscribe now. Don’t miss this offer before ends. Explore GPT Image 2 on AITOPIA.

GPT Image 2 is UNLIMITED FREE for new AITOPIA customers. For today’s launch only, customers who join and subscribe to a plan within the next 24 hours get GPT Image 2 UNLIMITED FREE in their first 4 hours. Access latest AI models and agents in one subscription on AITOPIA. Trusted by more than 3 million users, together with 150+ AI Agents, 500+ AI models. Subscribe now. Don’t miss this offer before ends. Explore GPT Image 2 on AITOPIA.

AITOPIA

265,761 views • 3 months ago

Watching Generalist AI demonstrate real-time intuition is incredible! 🤖✨ Check out this "Physical Commonsense" in action: the robot flips a washer, picks it up and then double-nudges it perfectly into a tight foam slot. This isn't a scripted motion; it’s an emergent behavior from a model trained on 270,000+ hours of real-world interaction. We are seeing the era where robots are starting to intuit and react to physics.

Watching Generalist AI demonstrate real-time intuition is incredible! 🤖✨ Check out this "Physical Commonsense" in action: the robot flips a washer, picks it up and then double-nudges it perfectly into a tight foam slot. This isn't a scripted motion; it’s an emergent behavior from a model trained on 270,000+ hours of real-world interaction. We are seeing the era where robots are starting to intuit and react to physics.

Humanoids daily

14,762 views • 6 months ago

Today we announced Gemini 2.0, our most capable AI model yet. With new advances in multimodality — like native image and audio output — and native tool use, it will enable us to build new AI agents that bring us closer to our vision of a universal assistant.

Today we announced Gemini 2.0, our most capable AI model yet. With new advances in multimodality — like native image and audio output — and native tool use, it will enable us to build new AI agents that bring us closer to our vision of a universal assistant.

Google

137,190 views • 1 year ago

Pi0 from Physical Intelligence is one of the best generalist Vision-Language-Action (VLA) models for robotics (and predecessor of Pi0.5). 🧵 Here's how it works + how you can fine-tune it on your own robot for a simple task:

Pi0 from Physical Intelligence is one of the best generalist Vision-Language-Action (VLA) models for robotics (and predecessor of Pi0.5). 🧵 Here's how it works + how you can fine-tune it on your own robot for a simple task:

Ilia

32,956 views • 11 months ago