Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

How can a visuomotor policy learn from internet videos? We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: paper:

ruoshi_liu's profile picture

Ruoshi Liu

2,800 subscribers

50,844 görüntüleme • 2 yıl önce •via X (Twitter)

EğitimSağlık & İyilikBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is? Checkout "Evaluating Robot Policies in a World Model". Paper: Demo: Code:
0:10

Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is? Checkout "Evaluating Robot Policies in a World Model". Paper: Demo: Code:

Sherry Yang

37,028 görüntüleme • 1 yıl önce

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.
0:14

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.

Ruihan Yang

58,891 görüntüleme • 1 yıl önce

Excited to introduce Diffusion Augmented Agents (DAAGs)✨. We give an agent control of a diffusion model, so it can create its own *synthetic experience*.🪄 The result is a lifelong agent that can learn new reward detectors and policies, much more efficiently. Here's how. 👇
0:18

Excited to introduce Diffusion Augmented Agents (DAAGs)✨. We give an agent control of a diffusion model, so it can create its own *synthetic experience*.🪄 The result is a lifelong agent that can learn new reward detectors and policies, much more efficiently. Here's how. 👇

Norman Di Palo

12,931 görüntüleme • 2 yıl önce

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.
0:20

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 görüntüleme • 8 ay önce

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:
0:14

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:

Lior Alexander

158,600 görüntüleme • 3 yıl önce

Super excited about Hydra-0 from Hongyu Li and team! The key idea is to use flow as a shared visual interface across embodiments/objects for controllable video generation, allowing a single generalist world model to learn from human, handheld-gripper, and robot interaction data. My favorite result is the video below: start from a real video of a human doing the task (left), extract the desired object flow, and condition the model on that flow (right). The model then hallucinates a plausible robot motion that could produce the same object motion. Very cool glimpse of how a generalist world model can bridge human demonstrations and robot control.
0:13

Super excited about Hydra-0 from Hongyu Li and team! The key idea is to use flow as a shared visual interface across embodiments/objects for controllable video generation, allowing a single generalist world model to learn from human, handheld-gripper, and robot interaction data. My favorite result is the video below: start from a real video of a human doing the task (left), extract the desired object flow, and condition the model on that flow (right). The model then hallucinates a plausible robot motion that could produce the same object motion. Very cool glimpse of how a generalist world model can bridge human demonstrations and robot control.

Yunzhu Li

10,850 görüntüleme • 1 ay önce

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:
0:20

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,795 görüntüleme • 2 yıl önce

Creating a "unified" policy for a robot is uniquely challenging for two reasons: (a) data is scarce; (b) a single robot can only act sequentially. In PolyTask, we learn a single policy to solve many tasks in a lifelong fashion given *just 1 demonstration* for each task. 🧵
0:17

Creating a "unified" policy for a robot is uniquely challenging for two reasons: (a) data is scarce; (b) a single robot can only act sequentially. In PolyTask, we learn a single policy to solve many tasks in a lifelong fashion given *just 1 demonstration* for each task. 🧵

Lerrel Pinto

46,427 görüntüleme • 2 yıl önce

The Dream 7B (diffusion reasoning language model) is OUT! 🚨 I built a demo so you can test it out (and check the diffusion process live) 𖣯🔍
0:10

The Dream 7B (diffusion reasoning language model) is OUT! 🚨 I built a demo so you can test it out (and check the diffusion process live) 𖣯🔍

apolinario (poli)

35,679 görüntüleme • 1 yıl önce

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.
0:10

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.

MrNeRF

24,729 görüntüleme • 1 yıl önce

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:
0:18

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,329 görüntüleme • 2 yıl önce

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -
0:15

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

Abhishek Gupta

11,430 görüntüleme • 1 yıl önce

Text-to-image diffusion transformer models learn to align text and image representations as a byproduct of their conditional denoising task. By taking the dot product between the text and image representations of a DiT model (like Flux 2), you can create rich saliency maps.
0:19

Text-to-image diffusion transformer models learn to align text and image representations as a byproduct of their conditional denoising task. By taking the dot product between the text and image representations of a DiT model (like Flux 2), you can create rich saliency maps.

Alec Helbling

94,195 görüntüleme • 9 ay önce

You pretrained a robot policy on millions of camera frames. Now you want to add a force sensor. Do you really have to retrain on everything from scratch? MuSe adds a force-torque sensor to a frozen vision-only policy using a tiny amount of contact data. - lifts contact-rich task success (peg insertion 60% to 87%, vase wiping 33% to 77%) - fuse the new sensor both early (shared token space) and late (cross-attention), train the policy as a world model that predicts future video, future force, and actions together, and replay old vision-only data with the force input masked to prevent forgetting. - adding new sensor modalities to pre-trained World Model can be cheap & improve performance significantly
0:14

You pretrained a robot policy on millions of camera frames. Now you want to add a force sensor. Do you really have to retrain on everything from scratch? MuSe adds a force-torque sensor to a frozen vision-only policy using a tiny amount of contact data. - lifts contact-rich task success (peg insertion 60% to 87%, vase wiping 33% to 77%) - fuse the new sensor both early (shared token space) and late (cross-attention), train the policy as a world model that predicts future video, future force, and actions together, and replay old vision-only data with the force input masked to prevent forgetting. - adding new sensor modalities to pre-trained World Model can be cheap & improve performance significantly

Vai Viswanathan

36,059 görüntüleme • 2 ay önce

We just did the largest live deployment of interactive robots in history. Users online could ask the robot to do things it was never taught to do. A true foundation model, with 0 task-specific data and 0 task fine tuning. 1/10
0:17

We just did the largest live deployment of interactive robots in history. Users online could ask the robot to do things it was never taught to do. A true foundation model, with 0 task-specific data and 0 task fine tuning. 1/10

Enigma

201,425 görüntüleme • 1 ay önce

So TeamYouTube has given me a warning and a strike in 24 hours for two separate videos under the "Harmful and Dangerous Acts" policy and "Harassment and Bullying" policy. I want you to just take a look at what was taken down with the timestamps TeamYouTube has provided A video where i am reacting to a video that is currently on their platform and a video where I'm saying "top 5 gooners" got taken down under this policy with both appeals getting rejected in under an hour TeamYouTube can i get a REAL HUMAN to look at these videos fairly and not a robot. Not one of those links saying "We have previously looked at the video and unfortunately it goes against our community guidelines" the appeal system does not work correctly.
0:22

So TeamYouTube has given me a warning and a strike in 24 hours for two separate videos under the "Harmful and Dangerous Acts" policy and "Harassment and Bullying" policy. I want you to just take a look at what was taken down with the timestamps TeamYouTube has provided A video where i am reacting to a video that is currently on their platform and a video where I'm saying "top 5 gooners" got taken down under this policy with both appeals getting rejected in under an hour TeamYouTube can i get a REAL HUMAN to look at these videos fairly and not a robot. Not one of those links saying "We have previously looked at the video and unfortunately it goes against our community guidelines" the appeal system does not work correctly.

Blueryai🌊

10,247 görüntüleme • 2 ay önce

If you are looking to learn WordPress website design in 2026 and cannot afford a paid course... Here are my top 3 YouTube channels where you can learn from.
0:13

If you are looking to learn WordPress website design in 2026 and cannot afford a paid course... Here are my top 3 YouTube channels where you can learn from.

Hecks || Website Designer

11,224 görüntüleme • 8 ay önce

i had a dream that i posted this and they had to add a dislike button to the website
0:16

i had a dream that i posted this and they had to add a dislike button to the website

user

307,936 görüntüleme • 8 ay önce

Predicting how the environment changes over time is a key task in robotics, enabling all kinds of downstream tasks. Trace Anything is a very interesting look at how you can train models which predict the trajectory each point will take in the future -- you can use this for robot manipulation, video prediction, and more. Learn more ->
0:16

Predicting how the environment changes over time is a key task in robotics, enabling all kinds of downstream tasks. Trace Anything is a very interesting look at how you can train models which predict the trajectory each point will take in the future -- you can use this for robot manipulation, video prediction, and more. Learn more ->

Chris Paxton

58,763 görüntüleme • 9 ay önce

Visual annotation with Ray3 brings fine tuned control to character expression. Create subtle motion, lift a brow, curve a smile, or shift a gaze. Direct emotions and from anger to love. Try Ray3 in Dream Machine.
0:19

Visual annotation with Ray3 brings fine tuned control to character expression. Create subtle motion, lift a brow, curve a smile, or shift a gaze. Direct emotions and from anger to love. Try Ray3 in Dream Machine.

Luma

397,787 görüntüleme • 11 ay önce