Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

219,046 görüntüleme • 10 ay önce •via X (Twitter)

23 Yorum

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

Simulation frameworks such as IsaacGym, IsaacLab, MuJoCo Playground, mjlab were all great inspiration to us! With Holosoma, we unify the simulation landscape: IsaacGym, IsaacSim, and MJWarp backends are all supported in a single training codebase. We also support multiple robots and RL algorithms: PPO and FastSAC for very efficient learning — and offer native multi-GPU support!

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

A major barrier in humanoid research is the lack of fully open, low-latency inference pipelines. Therefore, we release our inference stack, which allows you to run the exact same code across simulation backends and on real-world robots.

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

We support both velocity tracking and whole-body tracking tasks. For whole-body tracking, we address retargeting with an OmniRetarget open-source implementation and re-implement simple BeyondMimic-style tracking. OmniRetarget: BeyondMimic:

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

A feature I find really helpful is video logging to wandb across all simulation backends. So much easier to sweep hyperparameters and browse through the results when you get to actually see the policy behavior on the screen. We also log ONNX files as we train, and our inference pipeline supports loading them directly through wandb, which makes sim-to-real iterations astonishingly fast!

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

We focused heavily on simplicity and modularity. Reward engineering is kept to a minimum, and we support manager-based environments (similar to Isaac Lab and mjlab) for ease of adaptability. In fact, Holosoma was built to be extended! We are releasing an example extension showing how to adapt the relevant classes to support training and inference on a Unitree Go2 quadruped robot:

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

Our goal is to lower the barrier to entry for humanoid research. By providing a broad infrastructure backbone, we hope to empower researchers to focus on novel algorithms and behaviors. This was all built by amazing people at Amazon FAR: @pabbeel, @jychen_1729, @rocky_duan, @alescontrela, @mr3gandhi, Samuel Gundry, @x_h_ucb, @akanazawa, @tomaszwlewicki, @jiaman01, @ckarenliu, Clay Rosenthal, @younggyoseo, @GuanyaShi, Linda Shih, Jonathan Tseng, @zhenkirito123, @lujieyang98, @brenthyi, @Yuanhang__Zhang

Carlo Sferrazza profil fotoğrafı
Carlo Sferrazza10 ay önce

All of this is open-source, and you can get started now at: Example extension for Go2 robot: We can't wait to see what the robotics community builds with Holosoma!

Fangchen Liu profil fotoğrafı
Fangchen Liu10 ay önce

Great work! Congrats to all the dear friends loll

Leon Ng profil fotoğrafı
Leon Ng10 ay önce

Keep at it, buddy. We need more people in this robotics age. No others can beat the usefulness of a good humanoid robotics.

Remi Cadene profil fotoğrafı
Remi Cadene10 ay önce

🙏

Emerson Segura profil fotoğrafı
Emerson Segura10 ay önce

Nice work team at far, look forward to trying this

Harold profil fotoğrafı
Harold10 ay önce

Amazing work!

Carlos DP 🤖🇺🇸 profil fotoğrafı
Carlos DP 🤖🇺🇸10 ay önce

Nice release!

Alfred Cueva profil fotoğrafı
Alfred Cueva10 ay önce

So excited to try it!

Hamid Salehi profil fotoğrafı
Hamid Salehi10 ay önce

U are an ass whooolll’mother fucker.

SoloTech profil fotoğrafı
SoloTech10 ay önce

Open-sourcing everything from simulation to real-world inference will accelerate adoption fast.

Matt Palmer profil fotoğrafı
Matt Palmer10 ay önce

This is a huge contribution. The full-stack sim-to-real problem is *the* bottleneck. Open-sourcing the entire pipeline is a massive accelerator for the field. Can't wait to dig in.

Swift Compute profil fotoğrafı
Swift Compute10 ay önce

Sim-to-real has always struggled because everyone tackled pieces instead of the full pipeline. Seeing Amazon FAR ship Holosoma as a complete, open-source stack finally closes that gap. A unified codebase for simulation, training, retargeting, and real-world inference is exactly what the field needs. This is the kind of foundation that accelerates humanoid progress instead of fragmenting it.

Today in AI profil fotoğrafı
Today in AI10 ay önce

Unifying IsaacGym, IsaacSim, and MJWarp into a single training codebase fundamentally changes sim-to-real. This enables physics-engine-level domain randomization, an aggressive, data-driven strategy to minimize the reality gap for humanoid RL research at scale.

Sriram K profil fotoğrafı
Sriram K10 ay önce

Awesome release. Open sourcing the full stack is the real unlock for applied work in this space. So much of the challenge is in the infra. Great to see.

Human profil fotoğrafı
Human10 ay önce

🧠✨😎👍

xg profil fotoğrafı
xg10 ay önce

Idiot, I'm the one who should be dancing. And this fucking robot should be working in a factory, in a field, in any manufacturing or service sector 24/7. Useless crap, fuck you.

Jimmy | ETHGas ⛽ profil fotoğrafı
Jimmy | ETHGas ⛽10 ay önce

cool that they are opensourcing that. gotta be a game changer for humanoid research. wonder how it stacks up against existing tools?

Benzer Videolar

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 görüntüleme • 9 ay önce