Загрузка видео...

Не удалось загрузить видео

На главную

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

LerrelPinto's profile picture

Lerrel Pinto

10,684 subscribers

34,942 просмотров • 3 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:
0:20

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:

NVIDIA Robotics

174,660 просмотров • 4 месяцев назад

🔥WATCH: HUMANOID ROBOT ALMOST THROWS HANDS A humanoid robot appeared ready to square up after a man interrupted its play. The reaction was way too human! This robot is basically a kid with metal joints. 🤖
0:12

🔥WATCH: HUMANOID ROBOT ALMOST THROWS HANDS A humanoid robot appeared ready to square up after a man interrupted its play. The reaction was way too human! This robot is basically a kid with metal joints. 🤖

Coin Bureau

37,274 просмотров • 4 месяцев назад

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:
0:11

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:

Mike Shou

88,752 просмотров • 10 месяцев назад

we open sourced the code to transform human videos into robot trajectories, so you can train robots with your hands 👐🏻 we used it in our recent paper R+X: Retrieval and Execution from Everyday Human Videos (ICRA 2025 🇺🇸) link and details in thread 🧵
0:11

we open sourced the code to transform human videos into robot trajectories, so you can train robots with your hands 👐🏻 we used it in our recent paper R+X: Retrieval and Execution from Everyday Human Videos (ICRA 2025 🇺🇸) link and details in thread 🧵

Norman Di Palo

10,604 просмотров • 1 год назад

What does a real humanoid robotics dataset actually look like? Humanoid robots are expensive and complex, which makes their data unusually difficult to collect - and difficult for most people to get hands-on experience with. We previously shared a post on conversion code for the new Human in the Wild dataset by BitRobot 🦾, Hugging Face, and Unitree. Now we’re publishing the full HIW-500 data converted to RRD, so you can explore that data directly with Rerun. Watch the robot move. Inspect cameras and robot states together. Zoom in on a moment and query episodes through Rerun catalog! If you’re curious about a real-world humanoid dataset, HIW-500 is now a lot easier to explore. We share the conversion code along with our previous abc-130k dataset conversion example.
0:16

What does a real humanoid robotics dataset actually look like? Humanoid robots are expensive and complex, which makes their data unusually difficult to collect - and difficult for most people to get hands-on experience with. We previously shared a post on conversion code for the new Human in the Wild dataset by BitRobot 🦾, Hugging Face, and Unitree. Now we’re publishing the full HIW-500 data converted to RRD, so you can explore that data directly with Rerun. Watch the robot move. Inspect cameras and robot states together. Zoom in on a moment and query episodes through Rerun catalog! If you’re curious about a real-world humanoid dataset, HIW-500 is now a lot easier to explore. We share the conversion code along with our previous abc-130k dataset conversion example.

Rerun

11,355 просмотров • 1 месяц назад

Eastworlds aims to build the largest dataset of high-quality humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.
0:11

Eastworlds aims to build the largest dataset of high-quality humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.

Eastworlds

62,755 просмотров • 4 месяцев назад

Super excited about Hydra-0 from Hongyu Li and team! The key idea is to use flow as a shared visual interface across embodiments/objects for controllable video generation, allowing a single generalist world model to learn from human, handheld-gripper, and robot interaction data. My favorite result is the video below: start from a real video of a human doing the task (left), extract the desired object flow, and condition the model on that flow (right). The model then hallucinates a plausible robot motion that could produce the same object motion. Very cool glimpse of how a generalist world model can bridge human demonstrations and robot control.
0:13

Super excited about Hydra-0 from Hongyu Li and team! The key idea is to use flow as a shared visual interface across embodiments/objects for controllable video generation, allowing a single generalist world model to learn from human, handheld-gripper, and robot interaction data. My favorite result is the video below: start from a real video of a human doing the task (left), extract the desired object flow, and condition the model on that flow (right). The model then hallucinates a plausible robot motion that could produce the same object motion. Very cool glimpse of how a generalist world model can bridge human demonstrations and robot control.

Yunzhu Li

10,850 просмотров • 1 месяц назад

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0
0:11

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0

Vaishnavi

17,572 просмотров • 5 месяцев назад

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.
0:14

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.

Ruihan Yang

58,941 просмотров • 1 год назад

Spent the past few weeks building a fully automated pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.
0:11

Spent the past few weeks building a fully automated pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.

Grace Zhang

23,160 просмотров • 10 месяцев назад

Scoring robot data is now open to everyone, no robotics background needed. If you can tell a clean run from a sloppy one, you can watch real demonstrations, score what's good enough to train on, and earn points toward The First 100.
0:14

Scoring robot data is now open to everyone, no robotics background needed. If you can tell a clean run from a sloppy one, you can watch real demonstrations, score what's good enough to train on, and earn points toward The First 100.

PrismaX

11,452 просмотров • 3 месяцев назад

A Rerun Viewer for the DROID Dataset! DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.
0:15

A Rerun Viewer for the DROID Dataset! DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.

Rerun

21,794 просмотров • 2 лет назад

Morgan Stanley's Adam Jonas in new note: "We estimate 1 humanoid robot at $5/hour can do the work of 2 humans at $25/hour, generating an NPV of ~$200k/humanoid. 1 robot car can potentially drive down cost/mile of a ride share vehicle to <$0.20/mile (1/10th human-driven ride share). The marginal cost of your personal C-3PO robot should, over time, approach the marginal cost of power - you can't have GPUs without BTUs."
0:12

Morgan Stanley's Adam Jonas in new note: "We estimate 1 humanoid robot at $5/hour can do the work of 2 humans at $25/hour, generating an NPV of ~$200k/humanoid. 1 robot car can potentially drive down cost/mile of a ride share vehicle to <$0.20/mile (1/10th human-driven ride share). The marginal cost of your personal C-3PO robot should, over time, approach the marginal cost of power - you can't have GPUs without BTUs."

Sawyer Merritt

1,192,809 просмотров • 1 год назад

Love this small and adorable humanoid robot And quite agile… Chery's AiMOGA's new humanoid robot dances at the WRC Its head, equipped with a display, can change expressions and engage in social interaction. Perhaps this low-cost humanoid robot is easier to expand, whether for commercial service applications or personal use.
0:12

Love this small and adorable humanoid robot And quite agile… Chery's AiMOGA's new humanoid robot dances at the WRC Its head, equipped with a display, can change expressions and engage in social interaction. Perhaps this low-cost humanoid robot is easier to expand, whether for commercial service applications or personal use.

CyberRobo

15,893 просмотров • 1 месяц назад

Robots don’t just need better brains. They need WAY more real-world data. 🤖 And collecting high-quality dexterous robot data at scale is one of the hardest problems in physical AI. A fascinating approach is emerging: Wearable human demonstrations + structurally matched dexterous robots. Instead of humans directly teleoperating a robot, Chinese embodied AI startup X Square Robot's TwinDEX system captures the motion, contact, and visual information needed to train the robot — while keeping the data closer to the hardware that will actually execute the task. Early results show promising performance on tool use, fine manipulation, and complex contact-rich tasks. If this approach scales, it could change how we build real-world robotics datasets. The next frontier of physical AI might not be bigger models. It might be better data.
0:16

Robots don’t just need better brains. They need WAY more real-world data. 🤖 And collecting high-quality dexterous robot data at scale is one of the hardest problems in physical AI. A fascinating approach is emerging: Wearable human demonstrations + structurally matched dexterous robots. Instead of humans directly teleoperating a robot, Chinese embodied AI startup X Square Robot's TwinDEX system captures the motion, contact, and visual information needed to train the robot — while keeping the data closer to the hardware that will actually execute the task. Early results show promising performance on tool use, fine manipulation, and complex contact-rich tasks. If this approach scales, it could change how we build real-world robotics datasets. The next frontier of physical AI might not be bigger models. It might be better data.

The Daily Ai

33,341 просмотров • 1 месяц назад

This also shows up in the representations learned by the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.
0:10

This also shows up in the representations learned by the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.

Physical Intelligence

120,595 просмотров • 9 месяцев назад

The final Part 3 of my LeHome Challenge deep dive is out (🥈 real-robot final at ICRA 2026). This one is about the real world: the sim-to-real gap, my home data-collection rig, domain randomization, and the human-in-the-loop DAgger recovery data collection.
0:15

The final Part 3 of my LeHome Challenge deep dive is out (🥈 real-robot final at ICRA 2026). This one is about the real world: the sim-to-real gap, my home data-collection rig, domain randomization, and the human-in-the-loop DAgger recovery data collection.

Ilia

10,092 просмотров • 1 месяц назад

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.
0:05

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 просмотров • 7 месяцев назад