正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

Lerrel Pinto

9,633 subscribers

33,816 次观看 • 2 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:

NVIDIA Robotics

170,245 次观看 • 2 个月前

🔥WATCH: HUMANOID ROBOT ALMOST THROWS HANDS A humanoid robot appeared ready to square up after a man interrupted its play. The reaction was way too human! This robot is basically a kid with metal joints. 🤖

🔥WATCH: HUMANOID ROBOT ALMOST THROWS HANDS A humanoid robot appeared ready to square up after a man interrupted its play. The reaction was way too human! This robot is basically a kid with metal joints. 🤖

Coin Bureau

36,938 次观看 • 2 个月前

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:

Mike Shou

88,181 次观看 • 8 个月前

we open sourced the code to transform human videos into robot trajectories, so you can train robots with your hands 👐🏻 we used it in our recent paper R+X: Retrieval and Execution from Everyday Human Videos (ICRA 2025 🇺🇸) link and details in thread 🧵

we open sourced the code to transform human videos into robot trajectories, so you can train robots with your hands 👐🏻 we used it in our recent paper R+X: Retrieval and Execution from Everyday Human Videos (ICRA 2025 🇺🇸) link and details in thread 🧵

Norman Di Palo

10,604 次观看 • 1 年前

Eastworlds aims to build the largest dataset of high-quality humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.

Eastworlds aims to build the largest dataset of high-quality humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.

Eastworlds

62,755 次观看 • 2 个月前

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0

Vaishnavi

17,572 次观看 • 4 个月前

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.

Ruihan Yang

58,800 次观看 • 1 年前

Spent the past few weeks building a fully automated pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.

Spent the past few weeks building a fully automated pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.

Grace Zhang

23,124 次观看 • 8 个月前

A Rerun Viewer for the DROID Dataset! DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.

A Rerun Viewer for the DROID Dataset! DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.

Rerun

21,776 次观看 • 2 年前

Morgan Stanley's Adam Jonas in new note: "We estimate 1 humanoid robot at $5/hour can do the work of 2 humans at $25/hour, generating an NPV of ~$200k/humanoid. 1 robot car can potentially drive down cost/mile of a ride share vehicle to <$0.20/mile (1/10th human-driven ride share). The marginal cost of your personal C-3PO robot should, over time, approach the marginal cost of power - you can't have GPUs without BTUs."

Morgan Stanley's Adam Jonas in new note: "We estimate 1 humanoid robot at $5/hour can do the work of 2 humans at $25/hour, generating an NPV of ~$200k/humanoid. 1 robot car can potentially drive down cost/mile of a ride share vehicle to <$0.20/mile (1/10th human-driven ride share). The marginal cost of your personal C-3PO robot should, over time, approach the marginal cost of power - you can't have GPUs without BTUs."

Sawyer Merritt

1,192,665 次观看 • 1 年前

Love this small and adorable humanoid robot And quite agile… Chery's AiMOGA's new humanoid robot dances at the WRC Its head, equipped with a display, can change expressions and engage in social interaction. Perhaps this low-cost humanoid robot is easier to expand, whether for commercial service applications or personal use.

Love this small and adorable humanoid robot And quite agile… Chery's AiMOGA's new humanoid robot dances at the WRC Its head, equipped with a display, can change expressions and engage in social interaction. Perhaps this low-cost humanoid robot is easier to expand, whether for commercial service applications or personal use.

CyberRobo

15,492 次观看 • 6 天前

This also shows up in the representations learned by the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.

This also shows up in the representations learned by the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.

Physical Intelligence

120,595 次观看 • 8 个月前

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 次观看 • 6 个月前

Creating a "unified" policy for a robot is uniquely challenging for two reasons: (a) data is scarce; (b) a single robot can only act sequentially. In PolyTask, we learn a single policy to solve many tasks in a lifelong fashion given *just 1 demonstration* for each task. 🧵

Creating a "unified" policy for a robot is uniquely challenging for two reasons: (a) data is scarce; (b) a single robot can only act sequentially. In PolyTask, we learn a single policy to solve many tasks in a lifelong fashion given just 1 demonstration for each task. 🧵

Lerrel Pinto

46,427 次观看 • 2 年前

A student just sent a humanoid robot to collect his diploma — and I can’t stop thinking about it. In China, a teen built a robot that walked the stage, shook hands, and graduated for him. At first, it seemed funny. Then it felt like a wake-up call. We’re still teaching students to memorize — while they’re out there building the future. If a student can graduate through a robot, maybe it’s time schools start catching up. Innovation or rebellion — what do you think? #AI #Education #Innovation #Robotics #FutureOfWork #Technology

A student just sent a humanoid robot to collect his diploma — and I can’t stop thinking about it. In China, a teen built a robot that walked the stage, shook hands, and graduated for him. At first, it seemed funny. Then it felt like a wake-up call. We’re still teaching students to memorize — while they’re out there building the future. If a student can graduate through a robot, maybe it’s time schools start catching up. Innovation or rebellion — what do you think? #AI #Education #Innovation #Robotics #FutureOfWork #Technology

Pascal Bornet

229,853 次观看 • 8 个月前

The next manipulation tool may not live on your screen. It may stand in front of you look into your eyes and convince you that it understands. Would you rather face a humanoid robot strong enough to hurt you or one designed well enough to make you trust it? This robot copies blinking, eye contact, head movement and facial expressions to create the illusion of human presence. That may look impressive but it also opens a darker question. When a machine can look concerned appear friendly and imitate emotion people may start trusting signals that contain no real feeling no empathy and no moral responsibility. So which is more dangerous a robot with physical power or a robot that can manufacture trust? #HumanoidRobot #Robotics #AI

The next manipulation tool may not live on your screen. It may stand in front of you look into your eyes and convince you that it understands. Would you rather face a humanoid robot strong enough to hurt you or one designed well enough to make you trust it? This robot copies blinking, eye contact, head movement and facial expressions to create the illusion of human presence. That may look impressive but it also opens a darker question. When a machine can look concerned appear friendly and imitate emotion people may start trusting signals that contain no real feeling no empathy and no moral responsibility. So which is more dangerous a robot with physical power or a robot that can manufacture trust? #HumanoidRobot #Robotics #AI

Techniahqrobot | humanoid robots

11,650 次观看 • 1 个月前

Check out the new interactive humanoid robot, Yogi, introduced by Cartwheel Robotics. Yogi is designed as a companion for everyday life, offering help and companionship in settings from hospitals to living rooms. It features soft silicone skin for safe touch, precision actuators for predictable movement, and simple, natural interaction. The robot continuously learns new abilities for greater utility. Cartwheel's founder, Scott LaValley, has a deep background, having developed over four humanoid systems. His expertise includes nearly a decade developing the Atlas robot at Boston Dynamics, followed by five years at Disney leading the team that created the Groot robot. This extensive, hands-on experience is key to Cartwheel Robotics' innovations.

Check out the new interactive humanoid robot, Yogi, introduced by Cartwheel Robotics. Yogi is designed as a companion for everyday life, offering help and companionship in settings from hospitals to living rooms. It features soft silicone skin for safe touch, precision actuators for predictable movement, and simple, natural interaction. The robot continuously learns new abilities for greater utility. Cartwheel's founder, Scott LaValley, has a deep background, having developed over four humanoid systems. His expertise includes nearly a decade developing the Atlas robot at Boston Dynamics, followed by five years at Disney leading the team that created the Groot robot. This extensive, hands-on experience is key to Cartwheel Robotics' innovations.

RoboHub🤖

15,243 次观看 • 10 个月前

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more details

𝗥𝗼𝗯𝗼𝘁𝘀 𝗱𝗼𝗻’𝘁 𝗻𝗲𝗲𝗱 𝗺𝗼𝗿𝗲 𝗱𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻𝘀. 𝗧𝗵𝗲𝘆 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 — 𝗮𝗳𝘁𝗲𝗿 𝘄𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻𝘀. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: 👉 pretrain on human videos 👉 deploy robot policy 👉 observe failures 👉 reinterpret failures using human priors 👉 improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: 📈 40% → 81% success rate 🏆 Strong improvements over π0.6 RECAP and RISE ✔️ Zero human intervention during post-deployment improvement 🧬 Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same π0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. 📄 Paper: 🌐 Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zürich TU München Microsoft Check out Hanzhi's 🧵 for more details

Oier Mees

12,379 次观看 • 2 个月前