正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)

Yuke Zhu

22,462 subscribers

69,582 次观看 • 1 年前 •via X (Twitter)

科学技术教育新闻政治 #CoRL2024

Anya Rossi• Live Now

Private livecam show

10 条评论

Yuke Zhu 的头像

Yuke Zhu1 年前

OKAMI enables a humanoid robot to imitate a manipulation task given a single RGB-D human video. OKAMI first uses foundation models to identify task-relevant objects and reason about how they are manipulated, generating a reference manipulation plan for the robot to follow. (2/n)

Yuke Zhu 的头像

Yuke Zhu1 年前

Then, OKAMI synthesizes humanoid motions through what we call "object-aware retargeting," mapping the human motions onto the humanoid robot while adapting the new objects and their locations in test environments. (3/n)

Yuke Zhu 的头像

Yuke Zhu1 年前

We show that OKAMI can learn diverse manipulation skills, such as pick-and-place, pouring, pushing, articulate object interaction, and bimanual dexterous manipulation. (4/n)

Yuke Zhu 的头像

Yuke Zhu1 年前

Please see our paper and the project website for more: 📄 🌐 Joint work with @lijinhan21 @yifengzhu_ut Yuqi Xie @SteveTod1998 @kiwi_sherbet @geopavlakos

Adam Raudonis 的头像

Adam Raudonis1 年前

More like the billion dollar question :P

HyperLight Technologies ~ hyperlight.pls 的头像

HyperLight Technologies ~ hyperlight.pls1 年前

Wait till they learn martial arts...

Daniel R P de Melo 的头像

Daniel R P de Melo1 年前

This turns YouTube into pure oil 🛢️

ObotX - Open Source Robotics 的头像

ObotX - Open Source Robotics1 年前

q1. Are really super complex and expensive humanoid arms are legs needed for 99% of use cases? q2. Why videos are needed to simulate physics? :) Aircraft and spaceship construction doesnt neeed that :)

Greg Cook 的头像

Greg Cook1 年前

Great Q

Boris 的头像

Boris1 年前

Does the human learn and understand that he takes his own jobs? Even the engineer can be replaced. How intelligent is that.

相关视频

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot to real robots. Amazon FAR is the primary affiliation for the paper:

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot to real robots. Amazon FAR is the primary affiliation for the paper:

The Humanoid Hub

106,447 次观看 • 10 个月前

Humanoid robotics is entering a new phase： Xiaomi Robotics 0 (XR-0) explores learning manipulation directly from large-scale human videos — aligning video understanding with robot embodiment to generate executable actions. Watch humans->learn representations ->map to robot control. Robot data is scarce. Internet video is not. If video pretraining transfers reliably into embodied control, the scaling law of robotics changes. Deployment speed becomes a function of model alignment, not just hardware iteration. We’re moving from engineered skills to learned behavior. That’s a structural shift. project:

Humanoid robotics is entering a new phase： Xiaomi Robotics 0 (XR-0) explores learning manipulation directly from large-scale human videos — aligning video understanding with robot embodiment to generate executable actions. Watch humans->learn representations ->map to robot control. Robot data is scarce. Internet video is not. If video pretraining transfers reliably into embodied control, the scaling law of robotics changes. Deployment speed becomes a function of model alignment, not just hardware iteration. We’re moving from engineered skills to learned behavior. That’s a structural shift. project:

CyberRobo

18,356 次观看 • 5 个月前

.travis kalanick on why specialized robots beat humanoids for real work: "I'm not anti-humanoid, I'm non-humanoid. There are tasks that are good for humanoids, low scale, high diversity, in human environments, humanoid all the way. But the minute you have a machine doing a high-scale task, we're no longer in humanoid land, we're in specialized robotics land." "There was a moment nine months ago where humanoids were the mega rage of all time. When your hairdresser says he's getting into the humanoid business, we know something's wrong." "If I had to make 1,000 pancakes an hour, I'd never have a humanoid do it, you'd need 100 humanoids going gg-gg-gg. It's like the humanoid Olympics in Beijing, they're doing the marathon and it's cool, but all I could think is, imagine if you put wheels on them. Would you ever put a humanoid in a car to drive it?" "High scale, repetitive tasks do not need to be in human form, and they won't be, because humans are meant to do lots of things at low scale."

.travis kalanick on why specialized robots beat humanoids for real work: "I'm not anti-humanoid, I'm non-humanoid. There are tasks that are good for humanoids, low scale, high diversity, in human environments, humanoid all the way. But the minute you have a machine doing a high-scale task, we're no longer in humanoid land, we're in specialized robotics land." "There was a moment nine months ago where humanoids were the mega rage of all time. When your hairdresser says he's getting into the humanoid business, we know something's wrong." "If I had to make 1,000 pancakes an hour, I'd never have a humanoid do it, you'd need 100 humanoids going gg-gg-gg. It's like the humanoid Olympics in Beijing, they're doing the marathon and it's cool, but all I could think is, imagine if you put wheels on them. Would you ever put a humanoid in a car to drive it?" "High scale, repetitive tasks do not need to be in human form, and they won't be, because humans are meant to do lots of things at low scale."

MTS

52,573 次观看 • 10 天前

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥

Zhengyi “Zen” Luo

15,876 次观看 • 8 个月前

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data. How can we learn DEXTEROUS skills from them? In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.

Julen Urain

42,451 次观看 • 1 年前

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video

Helix is now learning directly from human video data We have already trained on data collected in the real world, including Brookfield residential units To our knowledge, this is the first instance of a humanoid robot learning navigation end-to-end using only human video

Figure

46,041 次观看 • 10 个月前

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

Ashish Kapoor

118,133 次观看 • 10 个月前

🤖💃 A Milestone in Human-Robot Collaboration We are proud to announce that UBTECH has been appointed as the Official Exclusive Humanoid Robot Partner of #ChainExpo 2026. To mark this partnership, our new-generation full-size commercial service humanoid robot Walker C1 performed breathtaking waltz and ballet with human dancers. This is more than a dance. It’s a glimpse into the future of intelligent service robotics — where full-scale humanoids work, interact, and move alongside us in perfect harmony. #UBTECH #WalkerC1 #HumanoidRobot #CommercialRobotics #AI #Waltz #Ballet #Dance

🤖💃 A Milestone in Human-Robot Collaboration We are proud to announce that UBTECH has been appointed as the Official Exclusive Humanoid Robot Partner of #ChainExpo 2026. To mark this partnership, our new-generation full-size commercial service humanoid robot Walker C1 performed breathtaking waltz and ballet with human dancers. This is more than a dance. It’s a glimpse into the future of intelligent service robotics — where full-scale humanoids work, interact, and move alongside us in perfect harmony. #UBTECH #WalkerC1 #HumanoidRobot #CommercialRobotics #AI #Waltz #Ballet #Dance

UBTECH Robotics

28,735 次观看 • 2 个月前

Twitter boi, do you want to see a humanoid AI lab from the inside? Well I do ;) Join me to visit Flexion A world-leading lab building the AI brains for humanoids – in Zurich, one of the best hubs for Physical AI. Their goal? Build the Android of robotics. One operating system that works on any humanoid hardware. It is crazy what their robots can do. We pushed a robot on the stairs. It didn't care. They left one in a rainy forest it was never trained on. Chilled and hiked there. And because they let robots train on their own, from zero through reinforcement learning, the robots do odd inhuman things. Eg they stand up from the ground in ways no human ever would. If their humanoid brain works, this will be industry defining. This is why Flexion is one of Europe's Most Ambitious Startups. 🇪🇺🔥 0:00 Intro 0:38 Inside a Lab with 14 Humanoids 2:17 What’s Actually Inside a Humanoid? 4:43 The Robot Navigates on Its Own 6:31 Why Grasping Is So Hard 7:01 Stairs, Pushes, and Balance Tests 7:55 Training 4,000 Robots in Parallel 10:50 Teleoperation vs Reinforcement Learning 14:35 The Case for Humanoids 15:55 Why Europe Could Win Robotics Please like & RT 🙏

Twitter boi, do you want to see a humanoid AI lab from the inside? Well I do ;) Join me to visit Flexion A world-leading lab building the AI brains for humanoids – in Zurich, one of the best hubs for Physical AI. Their goal? Build the Android of robotics. One operating system that works on any humanoid hardware. It is crazy what their robots can do. We pushed a robot on the stairs. It didn't care. They left one in a rainy forest it was never trained on. Chilled and hiked there. And because they let robots train on their own, from zero through reinforcement learning, the robots do odd inhuman things. Eg they stand up from the ground in ways no human ever would. If their humanoid brain works, this will be industry defining. This is why Flexion is one of Europe's Most Ambitious Startups. 🇪🇺🔥 0:00 Intro 0:38 Inside a Lab with 14 Humanoids 2:17 What’s Actually Inside a Humanoid? 4:43 The Robot Navigates on Its Own 6:31 Why Grasping Is So Hard 7:01 Stairs, Pushes, and Balance Tests 7:55 Training 4,000 Robots in Parallel 10:50 Teleoperation vs Reinforcement Learning 14:35 The Case for Humanoids 15:55 Why Europe Could Win Robotics Please like & RT 🙏

Andreas Klinger 🦾

14,761 次观看 • 4 个月前

Learning dexterous policies from human videos is challenging due to differences between human and robot hands. We present HuDOR, a method that learns dexterous policies within the robot's physical constraints using just one human video and an hour of online interactions! [1/n]

Learning dexterous policies from human videos is challenging due to differences between human and robot hands. We present HuDOR, a method that learns dexterous policies within the robot's physical constraints using just one human video and an hour of online interactions! [1/n]

Irmak Guzey

65,121 次观看 • 1 年前

🚨 BREAKING: German robot maker NEURA Robotics has raised €120 million in Series B funding to advance its cognitive robotics technology. Good to see the Europe robotics market moving! 🇪🇺 And yes I know this is news from mid-January, sometimes you have to be like Internet Explorer! Their product lineup includes collaborative arms, autonomous mobile robots, and humanoids. The funding will support new product development and the growth of its Neuraverse ecosystem, an operating system and marketplace for robotic skills. David Reger believes cognitive robots could become bigger than smartphones, with NEURA leading the way as Germany’s only humanoid robotics company. You can meet this and many other robots at the internationalhumanoidforum in Biel on the 20th & 21st of February! 🇨🇭 I also will be there, and I am happy to talk about the future of robotics!

🚨 BREAKING: German robot maker NEURA Robotics has raised €120 million in Series B funding to advance its cognitive robotics technology. Good to see the Europe robotics market moving! 🇪🇺 And yes I know this is news from mid-January, sometimes you have to be like Internet Explorer! Their product lineup includes collaborative arms, autonomous mobile robots, and humanoids. The funding will support new product development and the growth of its Neuraverse ecosystem, an operating system and marketplace for robotic skills. David Reger believes cognitive robots could become bigger than smartphones, with NEURA leading the way as Germany’s only humanoid robotics company. You can meet this and many other robots at the internationalhumanoidforum in Biel on the 20th & 21st of February! 🇨🇭 I also will be there, and I am happy to talk about the future of robotics!

Lukas Ziegler

46,521 次观看 • 1 年前

New Episode: Carolina Parada leads the robotics team at Google DeepMind. Carolina believes in a future with a broad, rich ecosystem of diverse robot types, where AI is smart enough to embody any robot. We discuss her journey, advancements in Gemini Robotics 1.5, cross-embodiment transfer, RL versus imitation, humanoids, world models, scaling laws, societal concerns, and more. 1:17 Introduction 1:38 Career journey and inspirations 3:50 Reaction to the 2022 ChatGPT launch 4:53 Google DeepMind's robotics mission 9:13 Key upgrades in Gemini Robotics 1.5 14:06 Agentic system's web usage 15:33 Robotics data-gathering methods 16:57 VLA vs. reasoning model training differences 18:22 Convergence of action and reasoning models 19:35 Cross-embodiment transfer 22:34 Learning directly from humans 24:20 Generalization challenges 27:01 Imitation versus reinforcement learning 28:51 Use of world models in robotics 30:31 Applications and testing of Gemini Robotics 1.5 33:04 Do humanoids deserve special focus? 35:06 Home humanoids timeline & limiting factors 37:19 Scaling laws in robotics versus self-driving 38:46 Value of learning classical robotics approaches 40:38 Ethical issues and societal impact

New Episode: Carolina Parada leads the robotics team at Google DeepMind. Carolina believes in a future with a broad, rich ecosystem of diverse robot types, where AI is smart enough to embody any robot. We discuss her journey, advancements in Gemini Robotics 1.5, cross-embodiment transfer, RL versus imitation, humanoids, world models, scaling laws, societal concerns, and more. 1:17 Introduction 1:38 Career journey and inspirations 3:50 Reaction to the 2022 ChatGPT launch 4:53 Google DeepMind's robotics mission 9:13 Key upgrades in Gemini Robotics 1.5 14:06 Agentic system's web usage 15:33 Robotics data-gathering methods 16:57 VLA vs. reasoning model training differences 18:22 Convergence of action and reasoning models 19:35 Cross-embodiment transfer 22:34 Learning directly from humans 24:20 Generalization challenges 27:01 Imitation versus reinforcement learning 28:51 Use of world models in robotics 30:31 Applications and testing of Gemini Robotics 1.5 33:04 Do humanoids deserve special focus? 35:06 Home humanoids timeline & limiting factors 37:19 Scaling laws in robotics versus self-driving 38:46 Value of learning classical robotics approaches 40:38 Ethical issues and societal impact

The Humanoid Hub

71,481 次观看 • 9 个月前

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data? Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵

Mahi Shafiullah 🏠🤖

93,934 次观看 • 1 个月前

Figure announces Project Go-Big: Internet-Scale Humanoid Pretraining and Direct Human-to-Robot Transfer The company has achieved a new milestone for the Helix AI model: after training exclusively on egocentric human data, Figure robots can now navigate real-world spaces from natural language commands such as “Go water the plants.” This also enabled a unified Helix model that can output both manipulation and navigation commands to the continuous control system. This ties into the Brookfield partnership that they announced yesterday. Brookfield is a global asset manager overseeing $1 trillion in assets, including 100,000 residential units, 500M square feet of commercial office space, and 160M square feet of logistics space. Using human video capture in a variety of Brookfield environments, Figure will amass critical pretraining data at scale, allowing them to explore the full potential of internet-scale embodied datasets for locomotion and manipulation.

Figure announces Project Go-Big: Internet-Scale Humanoid Pretraining and Direct Human-to-Robot Transfer The company has achieved a new milestone for the Helix AI model: after training exclusively on egocentric human data, Figure robots can now navigate real-world spaces from natural language commands such as “Go water the plants.” This also enabled a unified Helix model that can output both manipulation and navigation commands to the continuous control system. This ties into the Brookfield partnership that they announced yesterday. Brookfield is a global asset manager overseeing $1 trillion in assets, including 100,000 residential units, 500M square feet of commercial office space, and 160M square feet of logistics space. Using human video capture in a variety of Brookfield environments, Figure will amass critical pretraining data at scale, allowing them to explore the full potential of internet-scale embodied datasets for locomotion and manipulation.

The Humanoid Hub

32,879 次观看 • 10 个月前

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

Physical Intelligence

1,184,212 次观看 • 7 个月前

Skild has been quiet since emerging from stealth in July 2024. They just shared their journey so far, showcasing early milestones. Skild is building toward a single general-purpose, “omni-bodied” robotics foundation model: One Brain – any task, any robot. The Skild Brain can control humanoids, quadrupeds, or manipulators using a unified architecture trained on large-scale action datasets, including internet videos, simulation, and real-world data. Founded by CMU professors in 2023, the company has a strong group of researchers and engineers who have made some of the most seminal contributions to robotics research in the past decade. Now, they’re set to take it from the lab to the real world.

Skild has been quiet since emerging from stealth in July 2024. They just shared their journey so far, showcasing early milestones. Skild is building toward a single general-purpose, “omni-bodied” robotics foundation model: One Brain – any task, any robot. The Skild Brain can control humanoids, quadrupeds, or manipulators using a unified architecture trained on large-scale action datasets, including internet videos, simulation, and real-world data. Founded by CMU professors in 2023, the company has a strong group of researchers and engineers who have made some of the most seminal contributions to robotics research in the past decade. Now, they’re set to take it from the lab to the real world.

The Humanoid Hub

62,424 次观看 • 1 年前

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,208 次观看 • 8 个月前

ELON: AGI IS LIKELY TO EMERGE FROM OUR AUTONOMOUS CARS AND HUMANOIDS “We're not actually specifically focused on AGI. I'm simply saying that AGI seems likely to be an emergent property of what we're doing, because we're creating all these autonomous cars and autonomous humanoids that are actually a truly gigantic data stream that's coming in and being processed. It's by far the most amount of real-world data, and data you can't get by just searching the Internet, because you have to be out there in the world and interacting with people and interacting with the roads. Earth is a big place, and reality is messy and complicated. If you've got tens or hundreds of millions of autonomous vehicles and maybe even a comparable number of humanoids, maybe more than that, on the humanoid front, well, that's just the most amount of data. And if that video is being processed, it just seems likely that the cars will definitely get way better than human drivers, and the humanoid robots will become increasingly indistinguishable from humans, perhaps. And so then, like you said, you have this emergent property of AGI.” Source: Tesla AI Day, September 2022

ELON: AGI IS LIKELY TO EMERGE FROM OUR AUTONOMOUS CARS AND HUMANOIDS “We're not actually specifically focused on AGI. I'm simply saying that AGI seems likely to be an emergent property of what we're doing, because we're creating all these autonomous cars and autonomous humanoids that are actually a truly gigantic data stream that's coming in and being processed. It's by far the most amount of real-world data, and data you can't get by just searching the Internet, because you have to be out there in the world and interacting with people and interacting with the roads. Earth is a big place, and reality is messy and complicated. If you've got tens or hundreds of millions of autonomous vehicles and maybe even a comparable number of humanoids, maybe more than that, on the humanoid front, well, that's just the most amount of data. And if that video is being processed, it just seems likely that the cars will definitely get way better than human drivers, and the humanoid robots will become increasingly indistinguishable from humans, perhaps. And so then, like you said, you have this emergent property of AGI.” Source: Tesla AI Day, September 2022

Mario Nawfal

2,628,293 次观看 • 1 年前

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Excited to introduce COLA: Learning Human–Humanoid Coordination for Collaborative Object Carrying 🤝🤖 COLA makes humanoids truly helpful in human collaboration — capable of carrying objects, pushing carts, or responding to human push commands. It provides a proprioception-only policy for compliant human–humanoid coordination across diverse movement patterns. The core idea is simple yet effective: 👉 Fine-tune a collaborative policy from a locomotion policy using a residual teacher. 👉 Train in simulation and distill to a real-world student policy for deployment. Paper: Project:

Siyuan Huang

21,932 次观看 • 9 个月前

NVIDIA has published a paper on DREAMGEN – a powerful 4-step pipeline for generating synthetic data for humanoids that enables task and environment generalization. - Step 1: Fine-tune a video generation model using a small number of human teleoperation videos - Step 2: Prompt the fine-tuned model to turn a single real image into new AI-imagined videos - Step 3: Automatically label actions in the generated videos - Step 4: Train a robot AI model with the labeled synthetic dataset This enabled humanoid robots to perform 22 novel behaviors – such as pouring, opening/closing articulated objects, and manipulating a variety of tools. The original teleoperation dataset only included pick-and-place tasks. This takes task extensibility to another level without requiring human teleoperation for every single task. The pipeline will be made open-source soon. Project page:

NVIDIA has published a paper on DREAMGEN – a powerful 4-step pipeline for generating synthetic data for humanoids that enables task and environment generalization. - Step 1: Fine-tune a video generation model using a small number of human teleoperation videos - Step 2: Prompt the fine-tuned model to turn a single real image into new AI-imagined videos - Step 3: Automatically label actions in the generated videos - Step 4: Train a robot AI model with the labeled synthetic dataset This enabled humanoid robots to perform 22 novel behaviors – such as pouring, opening/closing articulated objects, and manipulating a variety of tools. The original teleoperation dataset only included pick-and-place tasks. This takes task extensibility to another level without requiring human teleoperation for every single task. The pipeline will be made open-source soon. Project page:

The Humanoid Hub

12,074 次观看 • 1 年前