Загрузка видео...
Не удалось загрузить видео
Happy Halloween from the Harvard Computational Robotics Group! 🎃🤖 Introducing GRASP: gradient-guided sampling for physics-based motion retargeting to dexterous hands and humanoids. Human demonstrations are a powerful source of robot skills. But kinematically retargeted motions can look right while requiring contacts and dynamics the robot cannot realize. Downstream RL... show more
20,336 просмотров • 17 дней назад •via X (Twitter)
Комментарии: 8

Building on our Tempered Sequential Monte Carlo (TSMC) work at RSS 2026, GRASP combines sampling-based exploration with gradient-guided refinement—without requiring differentiable simulator dynamics. The key idea: reuse rollouts we already collected, including rejected proposals, to fit local cost models and recover gradients for subsequent proposals. Even simple quadratic models provide useful guidance for MCMC rejuvenation. A Metropolis–Hastings correction uses the original simulator costs, preserving the intended cost-tilted target distribution for each fixed proposal kernel despite surrogate fitting errors. An MPC formulation extends the optimizer to long demonstrations. For example, GRASP achieves 94.2% success on 1,352 bimanual OakInk2 demonstrations with Sharpa hands, with a mean object position error of 2.87 cm. Crucially, the same tracking objective and optimizer settings work across Sharpa, Wuji, and XHand, without task-specific contact rewards or prescribed contact schedules. The optimizer discovers the contacts needed to reproduce the demonstrated motion. 2/3

A useful parallel is the recent SuperMimic work, which trains tracking policies by backpropagating through a learned world model. GRASP recovers trajectory-cost gradients from local models fitted to archived simulator rollouts, refining motion references before downstream policy learning. Both point to an opportunity we find exciting: extracting more useful optimization information from simulation experience. Paper, code, and demos: Great work by @yang_liu0922, @Tong_Wang7, and @AlvinBao85746! 3/3

Love your work! Grok explains why this is important in language an average nine year old can understand: "Robots can watch people do cool things, like spinning a pen or picking up a pumpkin, but just copying the moves often doesn’t work because the robot’s hands and body feel different forces and touch things in ways that don’t match. GRASP fixes those moves in a pretend world first so they actually make sense for the robot, which helps it learn the real skill much faster. This matters because it means robots can pick up lots of new tricks from videos or people without needing a human to carefully program every little contact and wiggle. The Halloween video shows robot hands writing “GRASP” and humanoid robots moving boxes more successfully after this fix, so the same idea can help with everyday stuff like carrying groceries or playing games. In short, it’s like giving robots better practice videos that already know how gravity and touching work, so they get better at using their hands and bodies for lots of different jobs instead of failing at the same things over and over."

Physics-aware retargeting is the step most human-video pipelines skip. In our UMI captures the bigger error is contact timing: when the gripper actually closes vs. when the fingers do. Does GRASP adjust timing, or only poses?

How much of the PPO tracking gain survives a change in object mass or friction at evaluation?

i watched the pen spin 12 times. i took notes. my hands are not that good. nice work, scientists.

open-loop replay working on real Sharpa hands is the strongest evidence here, means the refined refs are close to dynamically feasible

Fixing the reference before RL starts is a smart order of operations. A motion that looks right but needs contacts the hand cannot make just teaches the policy to chase the impossible. Pen spinning on Sharpa hands is a tough demo to pick, nice work from the lab.
