Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Generative models (diffusion/flow) are taking over robotics 🤖. But do we really need to model the full action distribution to control a robot? We suspected the success of Generative Control Policies (GCPs) might be "Much Ado About Noising." We rigorously tested the myths. 🧵👇

Chaoyi Pan

1,087 subscribers

111,108 views • 6 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Text-to-image generative models, meet robotics! We present ROSIE: Scaling RObot Learning with Semantically Imagined Experience, where we augment real robotics data with semantically imagined scenarios for downstream manipulation learning. Website: 🧵👇

Text-to-image generative models, meet robotics! We present ROSIE: Scaling RObot Learning with Semantically Imagined Experience, where we augment real robotics data with semantically imagined scenarios for downstream manipulation learning. Website: 🧵👇

Fei Xia

196,378 views • 3 years ago

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

Hila Chefer

67,393 views • 3 months ago

Generative #AI is creating new opportunities fast, but like-minded democracies need to get in control of the risks. As #G7 we took a big step forward today. We agreed draft guiding principles on generative AI for stakeholder consultation. We will 🔜 reach out in 🇪🇺! Stay tuned!

Generative #AI is creating new opportunities fast, but like-minded democracies need to get in control of the risks. As #G7 we took a big step forward today. We agreed draft guiding principles on generative AI for stakeholder consultation. We will 🔜 reach out in 🇪🇺! Stay tuned!

Věra Jourová

15,634 views • 2 years ago

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Abhishek Gupta

11,994 views • 1 year ago

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Heng Yang

18,968 views • 3 months ago

Excited to release FAST, our new robot action tokenizer! 🤖 Some highlights: - Simple autoregressive VLAs match diffusion VLA performance - Trains up to 5x faster - Works on all robot datasets we tested - First VLAs that work out-of-the-box in new environments! 🧵/

Excited to release FAST, our new robot action tokenizer! 🤖 Some highlights: - Simple autoregressive VLAs match diffusion VLA performance - Trains up to 5x faster - Works on all robot datasets we tested - First VLAs that work out-of-the-box in new environments! 🧵/

Karl Pertsch

90,582 views • 1 year ago

Chair Powell: "We haven't really started to see the effects of generative AI."

Chair Powell: "We haven't really started to see the effects of generative AI."

Yahoo Finance

14,679 views • 2 months ago

Google presents VLOGGER Multimodal Diffusion for Embodied Avatar Synthesis We propose VLOGGER, a method for audio-driven human video generation from a single input image of a person, which builds on the success of recent generative diffusion models. Our method consists of

AK

66,375 views • 2 years ago

Diffusion has shown great promise for generating robot **actions**, can it act as a **world model** to generate the future conditioned on actions? In our work led by han qi Haocheng Yin and in collaboration with Yilun Du, we show a **controllable** action-conditioned video diffusion model can produce photorealistic and (near) physics-accurate future predictions. This ability strengthens the policy via: - ranking different action proposals and selecting the best, or - **visual** trajectory optimization by optimizing the action proposals using gradient ascent. Learn more about Generative Predictive Control (GPC) at:

Diffusion has shown great promise for generating robot actions, can it act as a world model to generate the future conditioned on actions? In our work led by han qi Haocheng Yin and in collaboration with Yilun Du, we show a controllable action-conditioned video diffusion model can produce photorealistic and (near) physics-accurate future predictions. This ability strengthens the policy via: - ranking different action proposals and selecting the best, or - visual trajectory optimization by optimizing the action proposals using gradient ascent. Learn more about Generative Predictive Control (GPC) at:

Heng Yang

38,390 views • 1 year ago

🚨"Would you say we are currently under the control of the Reptilians?" "I think so. They control the media, they control the banks, they control the education system, they control religion.. What's left?" "If we could raise our Consciousness, not very much really, to the point we're we see what they're all about & understand what it's all about.. The Game is over, they can't win here!!" Consciousness is Key to this Reality.

🚨"Would you say we are currently under the control of the Reptilians?" "I think so. They control the media, they control the banks, they control the education system, they control religion.. What's left?" "If we could raise our Consciousness, not very much really, to the point we're we see what they're all about & understand what it's all about.. The Game is over, they can't win here!!" Consciousness is Key to this Reality.

UAPWixy

70,130 views • 3 months ago

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

103,815 views • 8 months ago

The missing piece for the ChatGPT moment of robotics is finally here Today, we are releasing phosphobot, an open-source app to control robots intuitively and train the most powerful AI models for the cheapest robots possible -> phospho-app/phosphobot 🧵 What we are building:

The missing piece for the ChatGPT moment of robotics is finally here Today, we are releasing phosphobot, an open-source app to control robots intuitively and train the most powerful AI models for the cheapest robots possible -> phospho-app/phosphobot 🧵 What we are building:

Pierre-Louis Biojout (PLB)

35,828 views • 1 year ago

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Physical Intelligence

116,845 views • 1 year ago

Farage ‘ We need to leave the ECHR to take back control’ Before that it was ‘We need to leave the Eu to take back control’ It will be a slow creep until ‘We need to have a dictator and martial law so we can take back control’ Facists handbook

Farage ‘ We need to leave the ECHR to take back control’ Before that it was ‘We need to leave the Eu to take back control’ It will be a slow creep until ‘We need to have a dictator and martial law so we can take back control’ Facists handbook

Roux the rescue 🏴󠁧󠁢󠁳󠁣󠁴󠁿🇨🇦

283,172 views • 1 year ago

To bring generalist intelligent robots to the real world, we have to overcome the data scarcity problem. At Rhoda, we are solving it by reformulating robot policies as video generation. Today, we introduce the Direct Video-Action Model (DVA)

To bring generalist intelligent robots to the real world, we have to overcome the data scarcity problem. At Rhoda, we are solving it by reformulating robot policies as video generation. Today, we introduce the Direct Video-Action Model (DVA)

Rhoda AI

68,121 views • 3 months ago

We are thrilled to announce the open-source release of Hallo2: a new avatar video generative model capable of generating stunning 4K resolution videos for up to 1 hour! 🎥✨ Explore the project for the open source code: Let's push the boundaries of avatar video generative model! #OpenSource #AI #FaceAnimation #GenerativeArt

Siyu ZHU

58,267 views • 1 year ago

We need an exit ramp off the highway to climate hell. We have control of the wheel. The 1.5 degree limit of global warming is still just about possible. But we need to fight harder. Now.

We need an exit ramp off the highway to climate hell. We have control of the wheel. The 1.5 degree limit of global warming is still just about possible. But we need to fight harder. Now.

António Guterres

235,099 views • 2 years ago

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,417 views • 8 months ago

Today, we are evolving Photoshop's Generative AI capabilities with Generative Expand. There is no need for selections; use the crop tool to expand the canvas, hit enter, and #AdobeFirefly will do the rest. Update or download the Photoshop (beta) now:

Today, we are evolving Photoshop's Generative AI capabilities with Generative Expand. There is no need for selections; use the crop tool to expand the canvas, hit enter, and #AdobeFirefly will do the rest. Update or download the Photoshop (beta) now:

Rufus Deuchler

162,244 views • 2 years ago

We need to get control of the mass immigration to our country. We need to start mass deportations. We need to be careful of who we bring into this country and have way more strict border control. We are not racist. We are not N@zis We don't hate. We're not violent. We're peaceful and kept it classy. We're about free speech. We're about freedom. We're about putting Canadians first. Until next time, stay tuned. We will be back!🇨🇦🫡

We need to get control of the mass immigration to our country. We need to start mass deportations. We need to be careful of who we bring into this country and have way more strict border control. We are not racist. We are not N@zis We don't hate. We're not violent. We're peaceful and kept it classy. We're about free speech. We're about freedom. We're about putting Canadians first. Until next time, stay tuned. We will be back!🇨🇦🫡

Joe Anidjar

45,597 views • 9 months ago