正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Amazon is training humanoids to move boxes. Makes sense! OmniRetarget is a data generation engine that enables complex loco-manipulation for humanoids. It uses offline retargeting from human MoCap datasets and augments data from single demos to produce 8 hours of trajectories that train RL policies in simulation, transferred zero-shot... show more

The Humanoid Hub

109,377 subscribers

106,447 次观看 • 8 个月前 •via X (Twitter)

科学技术游戏新闻政治

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Wow! Super impressive work by the new Amazon FAR team (from Covariant acquisition). Mapping long sequences of human motion (>30 sec) on robots with a differing shapes or interating with objects (box, table, etc) of different size. Enabling easier in-simulation data-augmentation and zero-shoot transfer. Super impressive and huge help to reduce the need for human teleop data (which is very complex to gather for humanoids) Dataset trajectories on Hugging Face (search OmniRetarget), full code framework to come soon Project page has some pretty three.js interactive demos

Wow! Super impressive work by the new Amazon FAR team (from Covariant acquisition). Mapping long sequences of human motion (>30 sec) on robots with a differing shapes or interating with objects (box, table, etc) of different size. Enabling easier in-simulation data-augmentation and zero-shoot transfer. Super impressive and huge help to reduce the need for human teleop data (which is very complex to gather for humanoids) Dataset trajectories on Hugging Face (search OmniRetarget), full code framework to come soon Project page has some pretty three.js interactive demos

Thomas Wolf

149,734 次观看 • 8 个月前

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 次观看 • 1 年前

Amazon’s training humanoids to carry boxes. I’m racking my brain over why they’d do that. ResMimic enables precise, expressive humanoid loco-manipulation, bridging gaps in general motion tracking (GMT) policies, which lack object awareness. Amazon FAR led the work with a multi-institutional team:

Amazon’s training humanoids to carry boxes. I’m racking my brain over why they’d do that. ResMimic enables precise, expressive humanoid loco-manipulation, bridging gaps in general motion tracking (GMT) policies, which lack object awareness. Amazon FAR led the work with a multi-institutional team:

The Humanoid Hub

94,752 次观看 • 8 个月前

This is how Amazon delivery bot will ring your doorbell. It is done by leveraging the same loco-manipulation framework that enables humanoid robots to move boxes, developed by Amazon's Frontier AI & Robotics team.

This is how Amazon delivery bot will ring your doorbell. It is done by leveraging the same loco-manipulation framework that enables humanoid robots to move boxes, developed by Amazon's Frontier AI & Robotics team.

The Humanoid Hub

196,178 次观看 • 8 个月前

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)

The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)

Yuke Zhu

69,574 次观看 • 1 年前

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 次观看 • 1 年前

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Why do most humanoid motion trackers still slip, skate, or clip through objects? Because the real bottleneck isn’t RL, it’s bad retargeting… OmniRetarget flips the script: instead of compensating with dozens of reward terms, it generates high-quality, interaction-preserving trajectories from human motions. The result is agile, human-like skills with minimal RL. ✅ 5 reward terms, 4 domain randomization terms, proprioception only, no history or curriculum ✅ Interaction mesh produces clean, contact-preserving motions with virtually no foot-skating ✅ Works across embodiments (Unitree H1, Booster T1) and scales from one demo to many scenes From carrying boxes to slope crawling to parkour-style rolls, policies trained on OmniRetarget data transfer zero-shot from sim to real hardware. Thanks for sharing, Zhen Wu! 📍Paper: Website: Dataset:

Ilir Aliu - eu/acc

61,491 次观看 • 8 个月前

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

Ashish Kapoor

118,084 次观看 • 9 个月前

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 次观看 • 8 个月前

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

Carlo Sferrazza

213,009 次观看 • 6 个月前

China Accelerates Deployment of Humanoid Robots in Real Scenarios. 🇨🇳 China is pushing humanoids and embodied AI into factories, logistics, retail, healthcare, safety, emergency response, and disaster prevention. The key is the real-robot data loop: train in real sites, collect task data, then use it to improve models, hardware components, safety, and deployment. The 2026 target is clear: 100+ high-value scenarios and the capability for 10,000-unit-scale rollout. For humanoids, this is the shift that matters: fewer staged moves, more useful work.

China Accelerates Deployment of Humanoid Robots in Real Scenarios. 🇨🇳 China is pushing humanoids and embodied AI into factories, logistics, retail, healthcare, safety, emergency response, and disaster prevention. The key is the real-robot data loop: train in real sites, collect task data, then use it to improve models, hardware components, safety, and deployment. The 2026 target is clear: 100+ high-value scenarios and the capability for 10,000-unit-scale rollout. For humanoids, this is the shift that matters: fewer staged moves, more useful work.

RoboHub🤖

15,568 次观看 • 5 天前

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 次观看 • 1 年前

HuB is a unified framework that enables humanoids to execute complex balancing tasks. It integrates reference motion refinement, balance-aware policy learning, and sim-to-real robustness training. Project page:

HuB is a unified framework that enables humanoids to execute complex balancing tasks. It integrates reference motion refinement, balance-aware policy learning, and sim-to-real robustness training. Project page:

The Humanoid Hub

53,251 次观看 • 1 年前

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

Jesse Zhang

99,840 次观看 • 3 个月前

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine

61,187 次观看 • 2 个月前

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

Siddhant Haldar

19,765 次观看 • 4 个月前

Okay, this is actually insane. 🪜 A humanoid that climbs a ladder — then performs tasks from it. Amazon FAR’s LadderMan learns ladder climbing in simulation, transfers zero-shot to a real Unitree G1, and supports stable on-ladder manipulation under tight ladder constraints. The demo: climb, stabilize, adjust a wall painting, hand off a box from a high shelf, tighten a light bulb. That last part matters because ladders are brutal for humanoids: sparse footholds, limited handholds, noisy depth, and almost no margin for balance errors. This is what real human spaces actually demand — not just flat floors.

Okay, this is actually insane. 🪜 A humanoid that climbs a ladder — then performs tasks from it. Amazon FAR’s LadderMan learns ladder climbing in simulation, transfers zero-shot to a real Unitree G1, and supports stable on-ladder manipulation under tight ladder constraints. The demo: climb, stabilize, adjust a wall painting, hand off a box from a high shelf, tighten a light bulb. That last part matters because ladders are brutal for humanoids: sparse footholds, limited handholds, noisy depth, and almost no margin for balance errors. This is what real human spaces actually demand — not just flat floors.

RoboHub🤖

11,041 次观看 • 9 天前

.Polymath is training world generation models to automate the creation of RL environments. Traditionally, RL environment generation has been bottlenecked by human data. Superintelligence will never be achieved by human data alone. Polymath is building the core technology to enable automated environment generation using far less human effort than traditionally required, and eventually none. This allows for more complex and realistic worlds, and higher quality, scale, and diversity of tasks. This will be essential to unlock RL scaling. The end goal is to create large-scale, long-horizon environments from a text description alone. This will enable the creation of worlds of arbitrary complexity and scale, which is foundational for training & evaluating autonomous, superintelligent AI agents. Congrats on the launch, Dylan Ma and Naren Yenuganti!

.Polymath is training world generation models to automate the creation of RL environments. Traditionally, RL environment generation has been bottlenecked by human data. Superintelligence will never be achieved by human data alone. Polymath is building the core technology to enable automated environment generation using far less human effort than traditionally required, and eventually none. This allows for more complex and realistic worlds, and higher quality, scale, and diversity of tasks. This will be essential to unlock RL scaling. The end goal is to create large-scale, long-horizon environments from a text description alone. This will enable the creation of worlds of arbitrary complexity and scale, which is foundational for training & evaluating autonomous, superintelligent AI agents. Congrats on the launch, Dylan Ma and Naren Yenuganti!

Y Combinator

44,192 次观看 • 3 个月前

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

185,867 次观看 • 2 个月前

Having humans annotate data to pre-train robots is expensive and time-consuming! Introducing SPRINT: A pre-training approach using LLMs and offline RL to equip robots w/ many language-annotated skills while minimizing human annotation effort! URL: 🧵👇

Having humans annotate data to pre-train robots is expensive and time-consuming! Introducing SPRINT: A pre-training approach using LLMs and offline RL to equip robots w/ many language-annotated skills while minimizing human annotation effort! URL: 🧵👇

Jesse Zhang

24,662 次观看 • 3 年前