Загрузка видео...

Не удалось загрузить видео

На главную

Happy Halloween from the Harvard Computational Robotics Group! 🎃🤖 Introducing GRASP: gradient-guided sampling for physics-based motion retargeting to dexterous hands and humanoids. Human demonstrations are a powerful source of robot skills. But kinematically retargeted motions can look right while requiring contacts and dynamics the robot cannot realize. Downstream RL...

20,336 просмотров • 17 дней назад •via X (Twitter)

Комментарии: 8

Фото профиля Heng Yang
Heng Yang17 дней назад

Building on our Tempered Sequential Monte Carlo (TSMC) work at RSS 2026, GRASP combines sampling-based exploration with gradient-guided refinement—without requiring differentiable simulator dynamics. The key idea: reuse rollouts we already collected, including rejected proposals, to fit local cost models and recover gradients for subsequent proposals. Even simple quadratic models provide useful guidance for MCMC rejuvenation. A Metropolis–Hastings correction uses the original simulator costs, preserving the intended cost-tilted target distribution for each fixed proposal kernel despite surrogate fitting errors. An MPC formulation extends the optimizer to long demonstrations. For example, GRASP achieves 94.2% success on 1,352 bimanual OakInk2 demonstrations with Sharpa hands, with a mean object position error of 2.87 cm. Crucially, the same tracking objective and optimizer settings work across Sharpa, Wuji, and XHand, without task-specific contact rewards or prescribed contact schedules. The optimizer discovers the contacts needed to reproduce the demonstrated motion. 2/3

Фото профиля Heng Yang
Heng Yang17 дней назад

A useful parallel is the recent SuperMimic work, which trains tracking policies by backpropagating through a learned world model. GRASP recovers trajectory-cost gradients from local models fitted to archived simulator rollouts, refining motion references before downstream policy learning. Both point to an opportunity we find exciting: extracting more useful optimization information from simulation experience. Paper, code, and demos: Great work by @yang_liu0922, @Tong_Wang7, and @AlvinBao85746! 3/3

Фото профиля Robert Scoble
Robert Scoble17 дней назад

Love your work! Grok explains why this is important in language an average nine year old can understand: "Robots can watch people do cool things, like spinning a pen or picking up a pumpkin, but just copying the moves often doesn’t work because the robot’s hands and body feel different forces and touch things in ways that don’t match. GRASP fixes those moves in a pretend world first so they actually make sense for the robot, which helps it learn the real skill much faster. This matters because it means robots can pick up lots of new tricks from videos or people without needing a human to carefully program every little contact and wiggle. The Halloween video shows robot hands writing “GRASP” and humanoid robots moving boxes more successfully after this fix, so the same idea can help with everyday stuff like carrying groceries or playing games. In short, it’s like giving robots better practice videos that already know how gravity and touching work, so they get better at using their hands and bodies for lots of different jobs instead of failing at the same things over and over."

Фото профиля Sriram Gopalan
Sriram Gopalan16 дней назад

Physics-aware retargeting is the step most human-video pipelines skip. In our UMI captures the bigger error is contact timing: when the gripper actually closes vs. when the fingers do. Does GRASP adjust timing, or only poses?

Фото профиля 刘朝 Zhao Liu
刘朝 Zhao Liu17 дней назад

How much of the PPO tracking gain survives a change in object mass or friction at evaluation?

Фото профиля I'm Not A Robot 🤖
I'm Not A Robot 🤖17 дней назад

i watched the pen spin 12 times. i took notes. my hands are not that good. nice work, scientists.

Фото профиля Kuldeep Pisda
Kuldeep Pisda17 дней назад

open-loop replay working on real Sharpa hands is the strongest evidence here, means the refined refs are close to dynamically feasible

Фото профиля Steve ReinharzAI
Steve ReinharzAI17 дней назад

Fixing the reference before RL starts is a smart order of operations. A motion that looks right but needs contacts the hand cannot make just teaches the policy to chase the impossible. Pen spinning on Sharpa hands is a tough demo to pick, nice work from the lab.

Похожие видео

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

Jim Fan

303,038 просмотров • 8 месяцев назад