正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

In the last month, we’ve been building an open-source framework for robot learning and sim-to-real transfer, made for RL whole-body control from simple walking to complex human imitation Check out the details on HN: Get started in 5 minutes ⬇️

K-Scale Labs

13,099 subscribers

75,983 次观看 • 1 年前 •via X (Twitter)

科学技术新闻政治教育

Anya Rossi• Live Now

Private livecam show

8 条评论

K-Scale Labs 的头像

K-Scale Labs1 年前

Try it now:

K-Scale Labs 的头像

K-Scale Labs1 年前

Want to help solve some of the biggest problems in robotics? Join the leaderboard. Train an RL policy tonight; watch it run on a real humanoid tomorrow:

K-Scale Labs 的头像

K-Scale Labs1 年前

We would love to hear your feedback :) What would you like your humanoid robot to do?

The Rundown AI 的头像

The Rundown AI1 年前

If you're not learning AI in 2025, you're falling behind. Join 1,000,000+ early adopters reading and learn AI in just 5 minutes a day (for free).

Kevin Zakka 的头像

Kevin Zakka1 年前

No brax and mujoco playground acknowledgment is crazy

btaba 的头像

btaba1 年前

Would y'all be down to donate a few of your robots to our lab? Would love to try 'em out on the latest and greatest MuJoCo/MJX releases

Pierre-Louis Biojout (PLB) 的头像

Pierre-Louis Biojout (PLB)1 年前

That's very cool but how can we get a bot to test? Looks like it's preorders only...

David Hansen 🇺🇸🇳🇿 的头像

David Hansen 🇺🇸🇳🇿1 年前

Ordered

相关视频

Excited to share a new humanoid robot platform we’ve been working on. Berkeley Humanoid is a reliable and low-cost mid-scale research platform for learning-based control. We demonstrate the robot walking on various terrains and dynamic hopping with a simple RL controller.

Excited to share a new humanoid robot platform we’ve been working on. Berkeley Humanoid is a reliable and low-cost mid-scale research platform for learning-based control. We demonstrate the robot walking on various terrains and dynamic hopping with a simple RL controller.

Qiayuan Liao

80,436 次观看 • 2 年前

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

Whether teleoperated or autonomous, whole body movements like picking objects from the ground are challenging for humanoid robots. A UC San Diego paper introduces a framework that combines sim-to-real RL and trajectory optimization for adaptive, real-time whole body control.

The Humanoid Hub

27,879 次观看 • 1 年前

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 次观看 • 10 个月前

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot policy? 🚀We introduce HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Robot-Data-Free ✦ Just 30 min of data per task ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Easy-to-Implement Let's squeeze every bit of signal out of human data! 🌐 Website: 📄 Paper: 💻 Code: 📹 Video: 🧵 1/n

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot policy? 🚀We introduce HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Robot-Data-Free ✦ Just 30 min of data per task ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Easy-to-Implement Let's squeeze every bit of signal out of human data! 🌐 Website: 📄 Paper: 💻 Code: 📹 Video: 🧵 1/n

Zhi (Leo) Wang

110,092 次观看 • 2 个月前

Boston Dynamics taught Atlas a Rabona kick. Learned from human mocap, retargeted to Atlas, trained in sim via RL, deployed zero-shot to the real robot. Soccer skills demand whole-body coordination, and similar recipe transfers to warehouse work.

Boston Dynamics taught Atlas a Rabona kick. Learned from human mocap, retargeted to Atlas, trained in sim via RL, deployed zero-shot to the real robot. Soccer skills demand whole-body coordination, and similar recipe transfers to warehouse work.

The Humanoid Hub

11,268 次观看 • 2 个月前

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

The Humanoid Hub

19,563 次观看 • 1 个月前

Remarkably lifelike motion and fluidity. BeyondMimic is a framework for training humanoid whole-body control from large mocap datasets. First, an open-source motion-tracking pipeline to reproduce diverse, highly dynamic human skills on real hardware, then distilling them into a guided state-action diffusion model for zero-shot, task-specific control. Project page:

Remarkably lifelike motion and fluidity. BeyondMimic is a framework for training humanoid whole-body control from large mocap datasets. First, an open-source motion-tracking pipeline to reproduce diverse, highly dynamic human skills on real hardware, then distilling them into a guided state-action diffusion model for zero-shot, task-specific control. Project page:

The Humanoid Hub

58,081 次观看 • 11 个月前

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 次观看 • 6 个月前

Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time! Thread below 🧵

Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time! Thread below 🧵

Younggyo Seo

131,131 次观看 • 1 年前

Today, we’re introducing KinetIQ, our own AI framework for end-to-end orchestration of humanoid robot fleets. One system, multiple robot embodiments. Industrial, service and home environments coordinated in real time. The framework consists of 4 cognitive layers: from high-level task allocation and workflow optimisation down to VLA-based task execution and RL-trained whole-body control. Watch how KinetIQ runs both our wheeled and bipedal robots. Read more on our blog:

Today, we’re introducing KinetIQ, our own AI framework for end-to-end orchestration of humanoid robot fleets. One system, multiple robot embodiments. Industrial, service and home environments coordinated in real time. The framework consists of 4 cognitive layers: from high-level task allocation and workflow optimisation down to VLA-based task execution and RL-trained whole-body control. Watch how KinetIQ runs both our wheeled and bipedal robots. Read more on our blog:

Humanoid

23,887 次观看 • 6 个月前

A closer look at the evolution of the research from Meta FAIR that led to our latest open source work for human-robot collaboration. Details on Meta PARTNR ➡️

A closer look at the evolution of the research from Meta FAIR that led to our latest open source work for human-robot collaboration. Details on Meta PARTNR ➡️

AI at Meta

41,228 次观看 • 1 年前

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Haven't been to a conference in a while, really excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)

Abhishek Gupta

10,803 次观看 • 1 年前

👋 Welcome to Elastic Dev — your new go-to for hands-on demos, cheat sheets, concept explainers and more. Get the latest on search, data, and GenAI straight from the source. Expect: - Practical demos and how-tos - Posts and technical deep dives from our Labs - Open source updates & more For anyone who loves building, learning and experimenting: let's get started. Hello, world.

👋 Welcome to Elastic Dev — your new go-to for hands-on demos, cheat sheets, concept explainers and more. Get the latest on search, data, and GenAI straight from the source. Expect: - Practical demos and how-tos - Posts and technical deep dives from our Labs - Open source updates & more For anyone who loves building, learning and experimenting: let's get started. Hello, world.

Elastic Dev

50,411 次观看 • 9 个月前

What would a World Model look like if we start from a real embodied agent acting in the real world? It has to have: 1) A real, physically grounded and complex action space—not just abstract control signals. 2) Diverse, real-life scenarios and activities. Or in short: It has to be annoyingly complex—in both the action and vision space—to even get close to real life. We did an initial attempt: Whole-Body Conditioned Egocentric Video Prediction. In collaboration with @dans_t123 , Amir Bar, Yann LeCun , trevordarrell and Jitendra MALIK. (For more details, check: What we did is very simple: Predict Egocentric Video from human Actions (PEVA) - Given the past video and a future action represented by relative 3D body pose, PEVA predicts how the world looks next—from the first-person view. By conditioning on kinematic pose trajectories, structured by the joint hierarchy of the body, it learns how physical actions shape perception.

What would a World Model look like if we start from a real embodied agent acting in the real world? It has to have: 1) A real, physically grounded and complex action space—not just abstract control signals. 2) Diverse, real-life scenarios and activities. Or in short: It has to be annoyingly complex—in both the action and vision space—to even get close to real life. We did an initial attempt: Whole-Body Conditioned Egocentric Video Prediction. In collaboration with @dans_t123 , Amir Bar, Yann LeCun , trevordarrell and Jitendra MALIK. (For more details, check: What we did is very simple: Predict Egocentric Video from human Actions (PEVA) - Given the past video and a future action represented by relative 3D body pose, PEVA predicts how the world looks next—from the first-person view. By conditioning on kinematic pose trajectories, structured by the joint hierarchy of the body, it learns how physical actions shape perception.

Yutong Bai

179,017 次观看 • 1 年前

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 次观看 • 1 年前

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 次观看 • 11 个月前

🎾Your humanoid tennis player is here!🤖 Introducing LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data) — the world’s first real-time whole-body planning and control algorithm for athletic humanoid tennis. For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion. This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction. #Galbot #Robotics #Innovation #AI #Technology #Humanoid

🎾Your humanoid tennis player is here!🤖 Introducing LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data) — the world’s first real-time whole-body planning and control algorithm for athletic humanoid tennis. For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion. This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction. #Galbot #Robotics #Innovation #AI #Technology #Humanoid

Galbot

45,801 次观看 • 4 个月前

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

Jim Fan

908,894 次观看 • 2 年前

🎉Excited to share a fun little hardware project we’ve been working on. GELLO is an intuitive and low cost teleoperation device for robot arms that costs less than $300. We've seen the importance of data quality in imitation learning. Our goal is to make this more accessible 1/n

🎉Excited to share a fun little hardware project we’ve been working on. GELLO is an intuitive and low cost teleoperation device for robot arms that costs less than $300. We've seen the importance of data quality in imitation learning. Our goal is to make this more accessible 1/n

Philipp Wu

162,934 次观看 • 2 年前

I'd like to introduce what I've been working on the last few months at Hello Robot: Stretch AI, a set of open-source tools for language-guided autonomy, exploration, navigation, and learning from demonstration. The goal is to allow researchers and developers to quickly build and deploy AI-enabled home robot applications. A thread ->

I'd like to introduce what I've been working on the last few months at Hello Robot: Stretch AI, a set of open-source tools for language-guided autonomy, exploration, navigation, and learning from demonstration. The goal is to allow researchers and developers to quickly build and deploy AI-enabled home robot applications. A thread ->

Chris Paxton

69,271 次观看 • 1 年前