Загрузка видео...

Не удалось загрузить видео

На главную

DiT4DiT is now open source! As the first humanoid-deployable Video-Action Model built on a world model, DiT4DiT continues to surprise us. In our paper last month, we showed its strong data efficiency. Now, with only slight modifications, it enables real-time whole-body autonomous pick-and-place. Paper: Code: Website:

ShuoYangAIR's profile picture

Shuo Yang

5,468 subscribers

15,969 просмотров • 6 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 5

Фото профиля Alfred Cueva
Alfred Cueva6 месяцев назад

Really interesting work! I am also trying to use VAM for humanoid whole body control

Фото профиля Michael Cho - Rbt/Acc
Michael Cho - Rbt/Acc6 месяцев назад

Really cool!

Фото профиля Tommy AutoTech
Tommy AutoTech6 месяцев назад

Insert me pitching world models to my VP instead of traditional vision systems. This pick-and-place fluidity is undeniable.

Фото профиля 𓅋 𐎫𐎤𐎶𐏀 ‎ﷺ
𓅋 𐎫𐎤𐎶𐏀 ‎ﷺ6 месяцев назад

Are you training in sim? How many episodes averagely per task?

Фото профиля Jie Wang
Jie Wang6 месяцев назад

very cool! Do you see an advantage of language steer-ability here?

Похожие видео

1/ Tencent releases HY World 1.5 (WorldPlay), an open-source real-time world model with 24fps A streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency In direct comparison to Genie 3, it's technically inferior, and there's no information on how long the consistency will last. However, Tencent has released a significant world model as open source for the first time, making it available to everyone. A few more examples, a thread 🧵
1:06

1/ Tencent releases HY World 1.5 (WorldPlay), an open-source real-time world model with 24fps A streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency In direct comparison to Genie 3, it's technically inferior, and there's no information on how long the consistency will last. However, Tencent has released a significant world model as open source for the first time, making it available to everyone. A few more examples, a thread 🧵

Chubby♨️

37,776 просмотров • 10 месяцев назад

We’re excited to share DiT4DiT, an end-to-end Video-Action Model for robot learning that unifies a video Diffusion Transformer and an action Diffusion Transformer in a single cascaded framework. By leveraging the rich spatiotemporal and physical dynamics learned through video generation, rather than static image-text priors, DiT4DiT achieves state-of-the-art results on LIBERO (98.6%) and RoboCasa GR1 (50.8%) with far less training data, delivering over 10× better sample efficiency and up to 7× faster convergence. Real-world deployment on a humanoid robot further shows robust generalization. We believe this is a step toward making video generation a powerful backbone for robot policy learning. This work builds upon the brilliant foundations laid by Nvidia's GR00T and Cosmos. Project: Paper: Code: Coming soon. In the meantime, you can ask your coding agent to reproduce the method based on GR00T/Cosmos.
2:32

We’re excited to share DiT4DiT, an end-to-end Video-Action Model for robot learning that unifies a video Diffusion Transformer and an action Diffusion Transformer in a single cascaded framework. By leveraging the rich spatiotemporal and physical dynamics learned through video generation, rather than static image-text priors, DiT4DiT achieves state-of-the-art results on LIBERO (98.6%) and RoboCasa GR1 (50.8%) with far less training data, delivering over 10× better sample efficiency and up to 7× faster convergence. Real-world deployment on a humanoid robot further shows robust generalization. We believe this is a step toward making video generation a powerful backbone for robot policy learning. This work builds upon the brilliant foundations laid by Nvidia's GR00T and Cosmos. Project: Paper: Code: Coming soon. In the meantime, you can ask your coding agent to reproduce the method based on GR00T/Cosmos.

Shuo Yang

31,872 просмотров • 7 месяцев назад

🔥Character AI in VR Space🔥 We present #SOLAMI, a social vision-language-action (VLA) model that enables 3D autonomous characters with *speech and body language* interaction - Project: - Paper Hugging Face : . Thanks AK !
1:43

🔥Character AI in VR Space🔥 We present #SOLAMI, a social vision-language-action (VLA) model that enables 3D autonomous characters with *speech and body language* interaction - Project: - Paper Hugging Face : . Thanks AK !

Ziwei Liu

27,957 просмотров • 1 год назад

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.
2:05

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 просмотров • 1 год назад

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:
0:20

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,881 просмотров • 2 лет назад

The Waypoint-1.5 technical paper is now live. Waypoint-1.5 is a real-time video diffusion world model designed to run on consumer GPUs, bringing interactive world models closer to practical, accessible deployment.
0:44

The Waypoint-1.5 technical paper is now live. Waypoint-1.5 is a real-time video diffusion world model designed to run on consumer GPUs, bringing interactive world models closer to practical, accessible deployment.

Overworld

81,687 просмотров • 3 месяцев назад

The first open-source unified world model for scalable robot manipulation: 5B-parameter open-source unified video-action world model that combines policy and world modeling to generate robot actions, predict future visuals, and evaluate task progress from observations, language, and state. The model is trained on 27.3K hours of heterogeneous data (17.8K real-robot teleop, 6.5K UMI demos, 3K egocentric human videos), enabling it to perform complex manipulation tasks like faucet connecting, bag packing, and toolbox storing as shown in demo videos. The approach supports test-time action refinement and points toward deployment-driven continuous improvement via fleet data. Thanks for sharing, Jianlan Luo (Jianlan Luo)! 📌 Resource links for τ0-WM: • Project page: • GitHub (code): • Hugging Face (model weights): • Paper (PDF): ——- Weekly robotics and AI insights. Subscribe free:
2:47

The first open-source unified world model for scalable robot manipulation: 5B-parameter open-source unified video-action world model that combines policy and world modeling to generate robot actions, predict future visuals, and evaluate task progress from observations, language, and state. The model is trained on 27.3K hours of heterogeneous data (17.8K real-robot teleop, 6.5K UMI demos, 3K egocentric human videos), enabling it to perform complex manipulation tasks like faucet connecting, bag packing, and toolbox storing as shown in demo videos. The approach supports test-time action refinement and points toward deployment-driven continuous improvement via fleet data. Thanks for sharing, Jianlan Luo (Jianlan Luo)! 📌 Resource links for τ0-WM: • Project page: • GitHub (code): • Hugging Face (model weights): • Paper (PDF): ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

28,276 просмотров • 4 месяцев назад

1/5 🚀 Thrilled to open-source OSCAR 🤖 — an action-conditioned world model for robotics, led by the visiting student in my group Zhuoyuan Wu! It generalizes across different robot embodiments with precise action controllability. All trained on a single GH200 GPU, and outperforms existing open-sourced baselines, which have larger model capacity and need more compute. Everything is public, including training data. 📄 Paper: 🌐 Project: 💻 Code: 🤗 Robot data: 🤗 Human data: 🤗 Weights: #Robotics #WorldModels #AI #OpenSource
0:31

1/5 🚀 Thrilled to open-source OSCAR 🤖 — an action-conditioned world model for robotics, led by the visiting student in my group Zhuoyuan Wu! It generalizes across different robot embodiments with precise action controllability. All trained on a single GH200 GPU, and outperforms existing open-sourced baselines, which have larger model capacity and need more compute. Everything is public, including training data. 📄 Paper: 🌐 Project: 💻 Code: 🤗 Robot data: 🤗 Human data: 🤗 Weights: #Robotics #WorldModels #AI #OpenSource

Jun Gao

105,237 просмотров • 4 месяцев назад

Our Puffin-World technical paper is now available. From one image or text prompt, it generates camera-controlled views with appearance, geometry and physical grounding, then reconstructs them into an explorable 3D world—all in one unified model. Paper: Blog: Dataset: Code:
1:56

Our Puffin-World technical paper is now available. From one image or text prompt, it generates camera-controlled views with appearance, geometry and physical grounding, then reconstructs them into an explorable 3D world—all in one unified model. Paper: Blog: Dataset: Code:

Chen Change Loy

10,439 просмотров • 1 месяц назад

Unitree Breakthrough: The World’s First Real-Time World Model-Driven Fully Autonomous Humanoid Robot Combat🥊
0:38

Unitree Breakthrough: The World’s First Real-Time World Model-Driven Fully Autonomous Humanoid Robot Combat🥊

Unitree

7,399,519 просмотров • 1 месяц назад

Google Genie 3 just got its biggest open-source rival 🤯 LingBot-World is an open-source world model that generates playable environments in real-time. → Built on Alibaba’s Wan2.2 → 10 minutes of stable generation → 16fps real-time interaction 100% Open Source.
2:16

Google Genie 3 just got its biggest open-source rival 🤯 LingBot-World is an open-source world model that generates playable environments in real-time. → Built on Alibaba’s Wan2.2 → 10 minutes of stable generation → 16fps real-time interaction 100% Open Source.

Alvaro Cintas

1,060,413 просмотров • 8 месяцев назад

Spatial Intelligence just went Open Source. 🌐 Introducing InSpatio-WorldFM — the world’s first open-source real-time interactive 3D world model. Last October, World Labs's RTFM showed the vision. Today, we’re bringing that idea into the open for everyone. ✅Efficient, multi-view consistent spatial reasoning ✅Real-time interactive exploration ✅Open-source weights & code Stop watching demos. Start building now. 👇
0:54

Spatial Intelligence just went Open Source. 🌐 Introducing InSpatio-WorldFM — the world’s first open-source real-time interactive 3D world model. Last October, World Labs's RTFM showed the vision. Today, we’re bringing that idea into the open for everyone. ✅Efficient, multi-view consistent spatial reasoning ✅Real-time interactive exploration ✅Open-source weights & code Stop watching demos. Start building now. 👇

InSpatio｜影溯

37,454 просмотров • 7 месяцев назад

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video
0:24

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video

Figure

46,041 просмотров • 1 год назад

Open-sourcing the whole package here! The last piece of our SONIC open-source, data collection, gr00t VLA post-training, inference just hit the repo! Train your Autonomous policies on G1 Whole-body with SONIC and gr00t N1.7! 🧑‍💻Code: 📑Docs:
0:39

Open-sourcing the whole package here! The last piece of our SONIC open-source, data collection, gr00t VLA post-training, inference just hit the repo! Train your Autonomous policies on G1 Whole-body with SONIC and gr00t N1.7! 🧑‍💻Code: 📑Docs:

Zhengyi “Zen” Luo

49,006 просмотров • 5 месяцев назад

🚀 We open-sourced LongLive — interactive, real-time long-video generation. 👥Generates video in real time as users enter text prompts. ⚡️20.7 FPS on a single H100,⏱️up to 240s per clip. 🎬Fine-tunes SOTA short-video models (e.g., Wan) into long-video generators. 🌍One step closer to World Models. All code for training & inference, model weights, demo page, and videos released! Paper: Code: Model: Demo Page: Introduction Video:
3:18

🚀 We open-sourced LongLive — interactive, real-time long-video generation. 👥Generates video in real time as users enter text prompts. ⚡️20.7 FPS on a single H100,⏱️up to 240s per clip. 🎬Fine-tunes SOTA short-video models (e.g., Wan) into long-video generators. 🌍One step closer to World Models. All code for training & inference, model weights, demo page, and videos released! Paper: Code: Model: Demo Page: Introduction Video:

Yukang Chen

11,943 просмотров • 1 год назад

1/4Introducing Qwen3-Coder-Next: Our latest open-weights model designed specifically to power the next generation of autonomous Coding Agents. Built on Qwen3-Next, this model is engineered to handle complex, long-horizon programming tasks with unprecedented efficiency. High-performance agentic intelligence is now in your hands.
2:01

1/4Introducing Qwen3-Coder-Next: Our latest open-weights model designed specifically to power the next generation of autonomous Coding Agents. Built on Qwen3-Next, this model is engineered to handle complex, long-horizon programming tasks with unprecedented efficiency. High-performance agentic intelligence is now in your hands.

Tongyi Lab

212,645 просмотров • 8 месяцев назад

We're thrilled to release & open-source Hunyuan3D World Model 1.0! This model enables you to generate immersive, explorable, and interactive 3D worlds from just a sentence or an image. It's the industry's first open-source 3D world generation model, compatible with CG pipelines for full editability & simulation. Set to transform game development, VR, digital content creation and so on. Get started now👇🏻 Project Page： Try it now： Github： Hugging Face：
1:24

We're thrilled to release & open-source Hunyuan3D World Model 1.0! This model enables you to generate immersive, explorable, and interactive 3D worlds from just a sentence or an image. It's the industry's first open-source 3D world generation model, compatible with CG pipelines for full editability & simulation. Set to transform game development, VR, digital content creation and so on. Get started now👇🏻 Project Page： Try it now： Github： Hugging Face：

Tencent Hy

1,231,203 просмотров • 1 год назад

This is the best way to understand how ML models actually work! Use Drawdata to draw a 2D dataset in Jupyter. Use it to actively pick data from the widget and update the model as the data is being drawn! Fully interactive, real-time, and open-source!
0:28

This is the best way to understand how ML models actually work! Use Drawdata to draw a 2D dataset in Jupyter. Use it to actively pick data from the widget and update the model as the data is being drawn! Fully interactive, real-time, and open-source!

Daily Dose of Data Science

52,132 просмотров • 1 год назад

We don’t generate videos. 🎬 We generate worlds from videos. 🌍 Introducing InSpatio-World — the world's first open-source real-time 4D world model‼️ Your input: a video clip Our output: a dynamic, navigable, persistent world 🕹️ explore freely across viewpoints ⏪ control time forward and backward 🔓 open-source and ready to build on :) Live demo: 🔗 Code & weights: 🔗 Project page: 🔗
1:32

We don’t generate videos. 🎬 We generate worlds from videos. 🌍 Introducing InSpatio-World — the world's first open-source real-time 4D world model‼️ Your input: a video clip Our output: a dynamic, navigable, persistent world 🕹️ explore freely across viewpoints ⏪ control time forward and backward 🔓 open-source and ready to build on :) Live demo: 🔗 Code & weights: 🔗 Project page: 🔗

InSpatio｜影溯

110,315 просмотров • 7 месяцев назад