Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🌎World models can predict, but controlling real robots from imagination sees a long-standing failure due to hallucination. 🧠Introducing Uncertainty-Aware RWM: a black-box, end-to-end neural dynamics model with long-horizon uncertainty propagation. 🎯

Chenhao Li

6,951 subscribers

54,751 views • 7 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🧠Model-Based RL shows promises but has seen limited success in real-world robotics. 🌎Introducing Robotic World Model, a black-box end-to-end neural dynamics model that bridges this gap, where policies are trained purely in imagination. NeurIPS Conference 🎯

🧠Model-Based RL shows promises but has seen limited success in real-world robotics. 🌎Introducing Robotic World Model, a black-box end-to-end neural dynamics model that bridges this gap, where policies are trained purely in imagination. NeurIPS Conference 🎯

Chenhao Li

106,524 views • 9 months ago

Do you want to obtain accurate, long-horizon predictions with neural PDE solvers? Introducing PDE-Refiner, a training process that improves rollout accuracies, offers uncertainty estimates and better data efficiency. 📜: 🖥️: 🧵1/9

Do you want to obtain accurate, long-horizon predictions with neural PDE solvers? Introducing PDE-Refiner, a training process that improves rollout accuracies, offers uncertainty estimates and better data efficiency. 📜: 🖥️: 🧵1/9

Phillip Lippe

26,027 views • 3 years ago

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025

Kaifeng Zhang

45,971 views • 1 year ago

Introducing LabOS². An early look at autonomous cell culture, as a long-horizon physical AI workflow for biomed. From protocol to real-world biological execution.

Introducing LabOS². An early look at autonomous cell culture, as a long-horizon physical AI workflow for biomed. From protocol to real-world biological execution.

AI4Science Catalyst

622,678 views • 3 months ago

⚡️ Introducing video world models that know when they don't know! 📽️ Action-conditioned video models can hallucinate a lot, generating videos that look plausible but are incorrect! How can we rigorously quantify their uncertainty, and know when to trust generations?

⚡️ Introducing video world models that know when they don't know! 📽️ Action-conditioned video models can hallucinate a lot, generating videos that look plausible but are incorrect! How can we rigorously quantify their uncertainty, and know when to trust generations?

Anirudha Majumdar

12,015 views • 8 months ago

We released τ₀-VLA, a robot foundation model built for long-horizon real-world manipulation. Robots can now clean rooms, cook meals, make drinks, collect laundry, and organize objects in autonomous episodes lasting up to 12 minutes. 🧵

We released τ₀-VLA, a robot foundation model built for long-horizon real-world manipulation. Robots can now clean rooms, cook meals, make drinks, collect laundry, and organize objects in autonomous episodes lasting up to 12 minutes. 🧵

Jianlan Luo

51,946 views • 1 month ago

Boston Dynamics has released a new video of their next-generation humanoid robot, Atlas. "We are excited to share some of our progress on developing Large Behavior Models for Atlas. This work is part of a collaboration between AI research teams at Toyota Research Institute (TRI) and Boston Dynamics. We have been building end-to-end language-conditioned policies that enable Atlas to accomplish long-horizon manipulation tasks."

Boston Dynamics has released a new video of their next-generation humanoid robot, Atlas. "We are excited to share some of our progress on developing Large Behavior Models for Atlas. This work is part of a collaboration between AI research teams at Toyota Research Institute (TRI) and Boston Dynamics. We have been building end-to-end language-conditioned policies that enable Atlas to accomplish long-horizon manipulation tasks."

Sawyer Merritt

200,744 views • 1 year ago

First look at our collaboration with Audi on bringing AI-driven robotics into industrial production. Our end-to-end pixel-to-action model, running on our bi-manual platform, is capable of performing a complex, dexterous and long-horizon insertion task.

First look at our collaboration with Audi on bringing AI-driven robotics into industrial production. Our end-to-end pixel-to-action model, running on our bi-manual platform, is capable of performing a complex, dexterous and long-horizon insertion task.

mimic

65,966 views • 6 months ago

We are introducing Felix. Felix is a purpose-built agent for high finance, designed for long-running, complex workflows and capable of producing decks, models, and documents end-to-end. Felix executes so you can focus where it matters.

We are introducing Felix. Felix is a purpose-built agent for high finance, designed for long-running, complex workflows and capable of producing decks, models, and documents end-to-end. Felix executes so you can focus where it matters.

Rogo

914,722 views • 4 months ago

Long video generation is a systems problem. Introducing LongLive-2.0 from NVIDIA Research: an end-to-end NVFP4 training and inference system for long video generation. Low-precision deployment often relies on post-training quantization, creating a gap between how models are trained and how they run. LongLive-2.0 aligns NVFP4-aware training, distillation, and W4A4 inference, maintaining strong benchmark quality while improving speed and memory efficiency.

Long video generation is a systems problem. Introducing LongLive-2.0 from NVIDIA Research: an end-to-end NVFP4 training and inference system for long video generation. Low-precision deployment often relies on post-training quantization, creating a gap between how models are trained and how they run. LongLive-2.0 aligns NVFP4-aware training, distillation, and W4A4 inference, maintaining strong benchmark quality while improving speed and memory efficiency.

NVIDIA AI

60,622 views • 3 months ago

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

Xiao Ma

46,323 views • 1 year ago

Chinese robotics company Astribot released their latest World-Action Model (WAM), Lumo-2. Technical breakdown: - based on a frozen 🥶 Qwen-3.5 4B VLM - trained in 3 progressive stages: 1. Action is aligned with latent world dynamics (an abstract representation of action). Real-world actions are anchored to physical constraints, while the latent space is guided to focus on motion-relevant changes. This bidirectional relationship makes the model physically grounded -> critical for a world model. 2. Action is aligned with vision and language. Reusing the vision backbone and action encoder from the frozen VLM, the authors add a custom vocabulary (for new actions), a semantic module, an action decoder, and an action projector. This aligns the (new) action representations with the (existing) vision-language semantic space. Most importantly: it builds a direct mapping from natural-language instructions to motor execution. 3. End-to-end training on language, video, and robot data. Only the new modules (everything outside the frozen backbone) are trained end-to-end across temporal reasoning, physical understanding, long-horizon, and dexterous manipulation. At the end of the day, Lumo-2 is not the best on benchmarks, but that's not the point. What's genuinely new: - a way to combine latent world modeling and action generation through progressive alignment - a physically-grounded latent dynamics space - it lifts performance on unseen objects using un-annotated human egocentric video + Vision Pro captures, no special transfer algorithm needed Why it matters: - the whole model is thin trainable adapters (semantic module, action decoder/projector) on a frozen 4B backbone (cheap) - that scale is suited for real-time embedded inference (~2.71× decode speedup, no accuracy loss) - its real moat is long-horizon execution, where the added temporal memory pays off far more than on any other task As a result, this robot can now make your latte (5x sped up video):

Chinese robotics company Astribot released their latest World-Action Model (WAM), Lumo-2. Technical breakdown: - based on a frozen 🥶 Qwen-3.5 4B VLM - trained in 3 progressive stages: 1. Action is aligned with latent world dynamics (an abstract representation of action). Real-world actions are anchored to physical constraints, while the latent space is guided to focus on motion-relevant changes. This bidirectional relationship makes the model physically grounded -> critical for a world model. 2. Action is aligned with vision and language. Reusing the vision backbone and action encoder from the frozen VLM, the authors add a custom vocabulary (for new actions), a semantic module, an action decoder, and an action projector. This aligns the (new) action representations with the (existing) vision-language semantic space. Most importantly: it builds a direct mapping from natural-language instructions to motor execution. 3. End-to-end training on language, video, and robot data. Only the new modules (everything outside the frozen backbone) are trained end-to-end across temporal reasoning, physical understanding, long-horizon, and dexterous manipulation. At the end of the day, Lumo-2 is not the best on benchmarks, but that's not the point. What's genuinely new: - a way to combine latent world modeling and action generation through progressive alignment - a physically-grounded latent dynamics space - it lifts performance on unseen objects using un-annotated human egocentric video + Vision Pro captures, no special transfer algorithm needed Why it matters: - the whole model is thin trainable adapters (semantic module, action decoder/projector) on a frozen 4B backbone (cheap) - that scale is suited for real-time embedded inference (~2.71× decode speedup, no accuracy loss) - its real moat is long-horizon execution, where the added temporal memory pays off far more than on any other task As a result, this robot can now make your latte (5x sped up video):

Léo

32,331 views • 1 month ago

World models hold a lot of promise for robotics, but they're data hungry and often struggle with long horizons. We learn models from a few (< 10) human demos that enable a robot to plan in completely novel scenes! Our key idea is to model *symbols* not pixels 👇

World models hold a lot of promise for robotics, but they're data hungry and often struggle with long horizons. We learn models from a few (< 10) human demos that enable a robot to plan in completely novel scenes! Our key idea is to model symbols not pixels 👇

Nishanth Kumar

82,931 views • 11 months ago

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 views • 2 years ago

Introducing FLUX-mimic, a next-generation Video-Action Model for general purpose dexterity, developed in partnership with Black Forest Labs. Late last year we published mimic-video and introduced Video-Action Models (VAM): a new family of robotics foundation models built on top of video generation models. We showed that robot control reduces to visual prediction, and that robot capability is downstream of improvements in video modeling accuracy. The obvious implication was that advances in the video modeling frontier would directly translate to increased capabilities in end-to-end robot learning. FLUX-mimic is that thesis at frontier scale: We've applied our VAM architecture to the strongest video backbone available today, FLUX 3 from Black Forest Labs, and trained it on data from our own robots and wearables. General-purpose dexterity, running on a single GPU on premises. Because the model already understands world dynamics, it needs far fewer demonstrations to learn a new task. This is game-changing for our mission to deploy robots to factory floors, where industrial robot data is scarce and expensive to collect. We're now testing and deploying FLUX-mimic with manufacturing leaders like Audi USA, on complex, multi-step manipulation long considered impossible for conventional automation.

Introducing FLUX-mimic, a next-generation Video-Action Model for general purpose dexterity, developed in partnership with Black Forest Labs. Late last year we published mimic-video and introduced Video-Action Models (VAM): a new family of robotics foundation models built on top of video generation models. We showed that robot control reduces to visual prediction, and that robot capability is downstream of improvements in video modeling accuracy. The obvious implication was that advances in the video modeling frontier would directly translate to increased capabilities in end-to-end robot learning. FLUX-mimic is that thesis at frontier scale: We've applied our VAM architecture to the strongest video backbone available today, FLUX 3 from Black Forest Labs, and trained it on data from our own robots and wearables. General-purpose dexterity, running on a single GPU on premises. Because the model already understands world dynamics, it needs far fewer demonstrations to learn a new task. This is game-changing for our mission to deploy robots to factory floors, where industrial robot data is scarce and expensive to collect. We're now testing and deploying FLUX-mimic with manufacturing leaders like Audi USA, on complex, multi-step manipulation long considered impossible for conventional automation.

mimic

117,541 views • 1 month ago

In a world filled with noise, speed, and uncertainty, we rarely pause long enough to truly connect with the present moment. In this free video series, Four Doorways to the Now, Eckhart Tolle invites you into four simple practices that bring calm, clarity, and meaning to everyday life. This is an invitation to slow down—not to escape the world, but to meet it from a deeper place. ✨ Sign up free:

In a world filled with noise, speed, and uncertainty, we rarely pause long enough to truly connect with the present moment. In this free video series, Four Doorways to the Now, Eckhart Tolle invites you into four simple practices that bring calm, clarity, and meaning to everyday life. This is an invitation to slow down—not to escape the world, but to meet it from a deeper place. ✨ Sign up free:

Eckhart Tolle

17,097 views • 11 months ago

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

OpenBMB

13,996 views • 7 months ago

Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early results in autonomous general-purpose dexterous capabilities – fast, reactive, smooth, precise, bi-manual coordinated sensorimotor control.

Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early results in autonomous general-purpose dexterous capabilities – fast, reactive, smooth, precise, bi-manual coordinated sensorimotor control.

Generalist

278,540 views • 1 year ago

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

AK

305,683 views • 3 years ago

Introducing PAN — MBZUAI’s New World Model for Interactive Intelligence Developed by MBZUAI’s Institute of Foundation Models, PAN is built for simulation, prediction, and agentic reasoning. Unlike traditional video generators that only output frames, PAN maintains a persistent internal state that evolves when guided with natural language. Its Generative Latent Prediction architecture combines: • A latent encoder to capture the world state • A dynamics module that evolves that state step-by-step • A video diffusion decoder that visualizes outcomes By decoding at every step using a causal sliding-window diffusion process, PAN stays grounded in real-world physics and maintains long-horizon continuity, a leap beyond single-shot models. Evaluated on action fidelity, long-horizon stability, and simulative planning, PAN delivers state-of-the-art performance compared to open models and rivals leading commercial systems. For robotics, autonomy, and decision support, PAN is a foundation for the next wave of intelligent, foresight-driven AI.

Introducing PAN — MBZUAI’s New World Model for Interactive Intelligence Developed by MBZUAI’s Institute of Foundation Models, PAN is built for simulation, prediction, and agentic reasoning. Unlike traditional video generators that only output frames, PAN maintains a persistent internal state that evolves when guided with natural language. Its Generative Latent Prediction architecture combines: • A latent encoder to capture the world state • A dynamics module that evolves that state step-by-step • A video diffusion decoder that visualizes outcomes By decoding at every step using a causal sliding-window diffusion process, PAN stays grounded in real-world physics and maintains long-horizon continuity, a leap beyond single-shot models. Evaluated on action fidelity, long-horizon stability, and simulative planning, PAN delivers state-of-the-art performance compared to open models and rivals leading commercial systems. For robotics, autonomy, and decision support, PAN is a foundation for the next wave of intelligent, foresight-driven AI.

MBZUAI

98,734 views • 9 months ago