Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Introducing 𝐃𝐫𝐞𝐚𝐦𝐆𝐞𝐧! We got humanoid robots to perform totally new 𝑣𝑒𝑟𝑏𝑠 in new environments through video world models. We believe video world models will solve the data problem in robotics. Bringing the paradigm of scaling human hours to GPU hours. Quick 🧵

Joel Jang

3,065 subscribers

117,493 просмотров • 1 год назад •via X (Twitter)

Наука и технологии Образование

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Joel Jang

Joel Jang1 год назад

Currently, robot data scaling is done through human labor. Recent work showed some potential signs of robots doing useful things in unseen homes (i.e., open-world generalization), but this required taking the physical robots and collecting data in 100+ homes.

Фото профиля Joel Jang

Joel Jang1 год назад

How about new 𝑣𝑒𝑟𝑏𝑠? When prompted, current robots stand still or perform tasks they were trained on (e.g. pick-and-place). There is currently no work in literature that can enable robots to perform new verbs outside of the teleoperation data for visuomotor robot policies.

Фото профиля Joel Jang

Joel Jang1 год назад

We introduce 𝐃𝐫𝐞𝐚𝐦𝐆𝐞𝐧, an embarrassingly simple 4-step pipeline that generates synthetic robot training data using video world models. (1) Fine-tune, (2) Prompt, (3) Extract actions, (4) Train. Simple as that.

Фото профиля Joel Jang

Joel Jang1 год назад

Through DreamGen, we generate “Dreams” or 𝑁𝑒𝑢𝑟𝑎𝑙 𝑇𝑟𝑎𝑗𝑒𝑐𝑡𝑜𝑟𝑖𝑒𝑠 of 22 new verbs in 10 unseen environments, and train robots to perform these tasks "zero-shot".

Фото профиля Joel Jang

Joel Jang1 год назад

DreamGen can also augment seen, contact-rich tasks that are hard to simulate (e.g. folding, scooping m&ms) for different robot systems (Franka & SO-100) and different robot policies (Diffusion Policy, π₀, GR00T N1), all resulting in non-trivial gains.

Фото профиля Joel Jang

Joel Jang1 год назад

We also introduce 𝘋𝑟𝘦𝑎𝘮𝐺𝘦𝑛 𝐵𝘦𝑛𝘤ℎ, a video generative benchmark for robotics that shows a positive correlation with downstream robot policies, so that video model researchers can help enable robotics without actually having to set up their own physical robot systems.

Фото профиля Joel Jang

Joel Jang1 год назад

This was an 8-month research effort, jointly led by @SeonghyeonYe @zy27962986 @szxiangjn, and advised by @scott_e_reed @yukez @DrJimFan , and with amazing collaborators from Nvidia GEAR Lab, Nvidia Cosmos Team, and UW. Check out more videos in the blog post & details in the paper! 🌐 Blog post: 📝 Paper: We will also be releasing the code & an API to try out DreamGen (prompt -> lerobot dataset) in the upcoming days! Do reach out if you are interested in joining our GR00T Dreams Team. We are scaling up the data, GPUs, and the team!

Фото профиля VistaShares

VistaShares1 год назад

Discover the future of AI investing. AIS delivers exposure to the companies driving the next wave of innovation—semiconductors, data centers, and AI applications. Explore the supercycle today.

Фото профиля Thomas Farfeleder

Thomas Farfeleder1 год назад

@threadreaderapp unroll

Фото профиля JulianSaks

JulianSaks1 год назад

So cool to see this! Congrats 🔥

Фото профиля Raymond Yu

Raymond Yu1 год назад

Congrats Joel! Doing big things👀

Похожие видео

🧵The quiet revolution in robotics isn't humanoid, or world models.. It's the rise of inspection & maintenance robots, quietly scaling across dirty, dangerous, and distant industrial environments. This sector is projected to hit $8.3B by 2030. Let's break down why it's growing so fast. [Save this thread for later 📌]

🧵The quiet revolution in robotics isn't humanoid, or world models.. It's the rise of inspection & maintenance robots, quietly scaling across dirty, dangerous, and distant industrial environments. This sector is projected to hit $8.3B by 2030. Let's break down why it's growing so fast. [Save this thread for later 📌]

Lukas Ziegler

171,961 просмотров • 2 месяцев назад

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Mahi Shafiullah 🏠🤖 RSS 2026 ✈️

92,706 просмотров • 29 дней назад

the micro1 robotics lab: real world data for intelligent models that co-exist in the physical world. we’re in-the-wild across 75 countries in 6,000+ unique environments collecting data. diverse movements, objects, and settings. the future of AI is as human as you can imagine. join us to start training robots today (link in comments).

the micro1 robotics lab: real world data for intelligent models that co-exist in the physical world. we’re in-the-wild across 75 countries in 6,000+ unique environments collecting data. diverse movements, objects, and settings. the future of AI is as human as you can imagine. join us to start training robots today (link in comments).

Ali Ansari

207,453 просмотров • 4 месяцев назад

Schaeffler is bringing UK's humanoid robots from concept to factory reality. Within the next 5 years, hundreds of humanoid robots will be deployed in Schaeffler's global factories, with beta versions first implemented in 2026–2027. Schaeffler is also becoming the preferred supplier of joint actuators for the humanoid wheeled platform and is jointly exploring next-generation actuator architectures with its partner. More importantly, the two companies will collaborate deeply on data and skills: continuously accumulating real-world manufacturing data through teleoperation, synthetic data, and peripheral data acquisition robots to train AI models, enabling humanoid robots to learn faster and perform more reliably in the factory.

Schaeffler is bringing UK's humanoid robots from concept to factory reality. Within the next 5 years, hundreds of humanoid robots will be deployed in Schaeffler's global factories, with beta versions first implemented in 2026–2027. Schaeffler is also becoming the preferred supplier of joint actuators for the humanoid wheeled platform and is jointly exploring next-generation actuator architectures with its partner. More importantly, the two companies will collaborate deeply on data and skills: continuously accumulating real-world manufacturing data through teleoperation, synthetic data, and peripheral data acquisition robots to train AI models, enabling humanoid robots to learn faster and perform more reliably in the factory.

CyberRobo

19,901 просмотров • 6 месяцев назад

Introducing General World Models. We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models. Learn more:

Introducing General World Models. We believe the next major advancement in AI will come from systems that understand the visual world and its dynamics, which is why we’re starting a new long-term research effort around general world models. Learn more:

Runway

714,292 просмотров • 2 лет назад

Proud to share what I’ve been working on with my colleagues at Rhoda AI: Direct Video-Action Models (DVA). TL;DR: - We pre-train causal video models from scratch to control robots - They handle complex production tasks for hours without intervention - Only use ~10 hours of robot data How? 🧵👇

Proud to share what I’ve been working on with my colleagues at Rhoda AI: Direct Video-Action Models (DVA). TL;DR: - We pre-train causal video models from scratch to control robots - They handle complex production tasks for hours without intervention - Only use ~10 hours of robot data How? 🧵👇

Tongzhou Mu 🤖🦾🦿

19,102 просмотров • 4 месяцев назад

Humanoid robots train for the first World Humanoid Robot Games taking place in Beijing, as part of an effort to push the boundaries of AI and robotics through sport

Humanoid robots train for the first World Humanoid Robot Games taking place in Beijing, as part of an effort to push the boundaries of AI and robotics through sport

Reuters

45,269 просмотров • 11 месяцев назад

Boston Dynamics and Google DeepMind announced a partnership to integrate Gemini Robotics AI models with the new electric Atlas humanoid robots. Got to say hi to Carolina Parada before the presentation

Boston Dynamics and Google DeepMind announced a partnership to integrate Gemini Robotics AI models with the new electric Atlas humanoid robots. Got to say hi to Carolina Parada before the presentation

The Humanoid Hub

14,689 просмотров • 6 месяцев назад

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

RoboPapers

27,567 просмотров • 5 месяцев назад

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

Ali Madani

80,642 просмотров • 1 год назад

Asimov (Asimov) pays households and businesses worldwide to record their daily routines, generating thousands of hours a day of diverse human movement data to train humanoid robots. They've built a global network, along with the full stack, from proprietary collection hardware to annotation tooling, and are already providing data to the largest robotics companies in the world. Congrats on the launch, Lyem Ningthou and Anshul!

Asimov (Asimov) pays households and businesses worldwide to record their daily routines, generating thousands of hours a day of diverse human movement data to train humanoid robots. They've built a global network, along with the full stack, from proprietary collection hardware to annotation tooling, and are already providing data to the largest robotics companies in the world. Congrats on the launch, Lyem Ningthou and Anshul!

Y Combinator

74,268 просмотров • 4 месяцев назад

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

309,942 просмотров • 1 год назад

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video

Figure

46,041 просмотров • 10 месяцев назад

State-of-the-art robot policies often need hundreds of hours of data. What if we needed none? Introducing TiPToP: a manipulation system that zero-shots open-world tasks from pixels and language using vision foundation models and GPU-parallelized Task and Motion Planning (TAMP).

State-of-the-art robot policies often need hundreds of hours of data. What if we needed none? Introducing TiPToP: a manipulation system that zero-shots open-world tasks from pixels and language using vision foundation models and GPU-parallelized Task and Motion Planning (TAMP).

Nishanth Kumar

78,298 просмотров • 4 месяцев назад

Netanyahu: There is one challenge in particular—what I call the "Manhattan Project." We will be the first in the world to solve the problem of explosive drones. It is a global problem. But we are going to solve it.

Netanyahu: There is one challenge in particular—what I call the "Manhattan Project." We will be the first in the world to solve the problem of explosive drones. It is a global problem. But we are going to solve it.

Clash Report

22,432 просмотров • 23 дней назад

The biggest challenge in robotics isn’t building better robots—it’s getting better data. Anastasis Germanidis, co-founder and CEO of Runway, on how world models could break the cycle.

The biggest challenge in robotics isn’t building better robots—it’s getting better data. Anastasis Germanidis, co-founder and CEO of Runway, on how world models could break the cycle.

TIME

11,114,604 просмотров • 3 месяцев назад

Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog →

Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog →

LeRobot

281,356 просмотров • 3 месяцев назад

To bring generalist intelligent robots to the real world, we have to overcome the data scarcity problem. At Rhoda, we are solving it by reformulating robot policies as video generation. Today, we introduce the Direct Video-Action Model (DVA)

To bring generalist intelligent robots to the real world, we have to overcome the data scarcity problem. At Rhoda, we are solving it by reformulating robot policies as video generation. Today, we introduce the Direct Video-Action Model (DVA)

Rhoda AI

70,151 просмотров • 4 месяцев назад

Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning. The GR00T model will enable a robot to understand multimodal instructions, such as language, video, and demonstration, and perform a variety of useful tasks. We are collaborating with many leading humanoid companies around the world, so that GR00T may transfer across embodiments and help the ecosystem thrive. GR00T is born on NVIDIA’s deep technology stack. We simulate in Isaac Lab (new app on Omniverse Isaac Sim for humanoid learning), train on OSMO (new compute orchestration system to scale up models), and deploy to Jetson Thor (new edge GPU chip designed to power GR00T). Announced in Jensen's keynote, Project GR00T is a cornerstone for the “Foundation Agent” roadmap of the newly founded GEAR Lab. At GEAR, we are building generally capable agents that learn to act skillfully in many worlds, virtual and real. See if you can spot "GEAR" in the video ;) Join us on the journey to land on the moon.

Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning. The GR00T model will enable a robot to understand multimodal instructions, such as language, video, and demonstration, and perform a variety of useful tasks. We are collaborating with many leading humanoid companies around the world, so that GR00T may transfer across embodiments and help the ecosystem thrive. GR00T is born on NVIDIA’s deep technology stack. We simulate in Isaac Lab (new app on Omniverse Isaac Sim for humanoid learning), train on OSMO (new compute orchestration system to scale up models), and deploy to Jetson Thor (new edge GPU chip designed to power GR00T). Announced in Jensen's keynote, Project GR00T is a cornerstone for the “Foundation Agent” roadmap of the newly founded GEAR Lab. At GEAR, we are building generally capable agents that learn to act skillfully in many worlds, virtual and real. See if you can spot "GEAR" in the video ;) Join us on the journey to land on the moon.

Jim Fan

1,076,844 просмотров • 2 лет назад

Google DeepMind introduced two foundational models for embodied reasoning, enabling robots to comprehend, react, and take action in the physical world: ⦿ Gemini Robotics – built on Gemini 2.0. Integrates vision, language, and action for real-world dexterity, . ⦿ Gemini Robotics-ER – Enhances spatial reasoning for advanced robotic control. They are working with Apptronik to develop the next generation of humanoid robots.

Google DeepMind introduced two foundational models for embodied reasoning, enabling robots to comprehend, react, and take action in the physical world: ⦿ Gemini Robotics – built on Gemini 2.0. Integrates vision, language, and action for real-world dexterity, . ⦿ Gemini Robotics-ER – Enhances spatial reasoning for advanced robotic control. They are working with Apptronik to develop the next generation of humanoid robots.

The Humanoid Hub

73,097 просмотров • 1 год назад