Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

Wenli Xiao

2,600 subscribers

70,332 Aufrufe • vor 2 Jahren •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Wenli Xiao

Wenli Xiaovor 2 Jahren

Please checkout our website for more videos! Website: Paper Link: Video: Team: @ChongZitaZhang @_wenlixiao @TairanHe99 @GuanyaShi Thanks for the help from: @arthurallshire @JasonJZLiu @MiladShafieeA @guanqi_he Justin Macey and Jessica Hodgins

Profilbild von Chaoyi Pan

Chaoyi Panvor 2 Jahren

Congrats @_wenlixiao!

Profilbild von Wenli Xiao

Wenli Xiaovor 2 Jahren

Thank you Chaoyi!

Profilbild von Haoru Xue

Haoru Xuevor 2 Jahren

Congratulations! @_wenlixiao

Profilbild von Wenli Xiao

Wenli Xiaovor 2 Jahren

Thank you Haoru!

Profilbild von Milad Shafiee

Milad Shafieevor 2 Jahren

Nice to see doggy learned to play😁 nice work 🙌

Profilbild von Shiqi Yang

Shiqi Yangvor 2 Jahren

Congrats!

Profilbild von Wenli Xiao

Wenli Xiaovor 2 Jahren

Thanks Shiqi, your 📺Open-Television is also very impressive, we should collaborate someday🦾

Profilbild von Chen Tessler

Chen Tesslervor 2 Jahren

Awesome!

Profilbild von Mikhail Kakanov

Mikhail Kakanovvor 2 Jahren

Looks really cool 👍

Ähnliche Videos

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

The Humanoid Hub

19,563 Aufrufe • vor 14 Tagen

X-Humanoid just dropped TG-VLA — the first full-size, whole-body VLA framework for humanoids. 🤖 Most VLA demos today still look arm-centric: see, plan, reach, grasp. The legs are mostly there to carry the arms. TG-VLA pushes action into the whole body: → HEX: task context & cross-embodiment learning → HAF-VLA: high-DoF motion → structured action flows → DSRL-DCT: online RL through a compressed latent space The real shift isn't a smarter hand — it's locomotion, torso control, balance, and manipulation inside one control loop. That's the move: from mobile dual-arm machines to full-body agents.

X-Humanoid just dropped TG-VLA — the first full-size, whole-body VLA framework for humanoids. 🤖 Most VLA demos today still look arm-centric: see, plan, reach, grasp. The legs are mostly there to carry the arms. TG-VLA pushes action into the whole body: → HEX: task context & cross-embodiment learning → HAF-VLA: high-DoF motion → structured action flows → DSRL-DCT: online RL through a compressed latent space The real shift isn't a smarter hand — it's locomotion, torso control, balance, and manipulation inside one control loop. That's the move: from mobile dual-arm machines to full-body agents.

RoboHub🤖

35,528 Aufrufe • vor 14 Tagen

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

Ashish Kapoor

118,133 Aufrufe • vor 10 Monaten

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi

Zhengyi “Zen” Luo

47,305 Aufrufe • vor 2 Jahren

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 Aufrufe • vor 11 Monaten

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia @ ICML 🇰🇷 tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia @ ICML 🇰🇷 tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

RoboPapers

44,794 Aufrufe • vor 7 Monaten

LimX Dynamics just released LimX Luna, officially described by the company as the first mass-deliverable full-size female humanoid robot. 🤖 This is not aimed at warehouses; it is built for malls, theme parks, auto shows, brand events, and other commercial spaces where attention is the product. Luna stands 160 cm tall, has 27 body DoF excluding end effectors, and combines whole-body motion control with voice, vision, facial expressions, gesture interaction, and motion imitation. The sharper part is the content stack: users can upload dance videos, build scenes with zero-code tools in LimX Studio, and run synchronized shows with up to 200 robots. That makes Luna less like a one-off demo machine and more like a programmable entertainment platform for foot traffic, performances, and social clips. For humanoids, this is a useful shift: not only “can it do labor,” but “can it create a consumer experience people actually stop to watch.”

RoboHub🤖

13,909 Aufrufe • vor 1 Monat

How humanoids move from "Kung Fu" mode to "Blue-collar" work 🤖🦾 Unitree founder Xingxing Wang just dropped some major truth bombs about the reality of robot labor at the HEIS meeting in Beijing Extreme mobility like the "Kung Fu" bot at the Gala is the prerequisite—stable movement is the foundation for robots to actually start working Unitree has been deploying humanoids at NIO and Geely since 2024 with a 100% success rate on single tasks like simple parts assembly The real challenge is long-sequence complex tasks with 20+ steps—success rates struggle here especially when handling tiny parts Whole-body teleoperation is the key technical breakthrough for capturing high-quality physical world data to solve the industry's data scarcity Standardization is now the priority—setting unified task metrics, evaluation systems, and safety baselines to move beyond one-off lab innovations In 2025 Unitree shipped over 5,500 humanoids with more than 6,500 units coming off the production line leading the global market in volume The end goal is clear—Unitree wants humanoids working in its own factories to manufacture even more robots, closing the automation loop Source: HEIS Annual Meeting #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #Unitree #SmartManufacturing #Automation

How humanoids move from "Kung Fu" mode to "Blue-collar" work 🤖🦾 Unitree founder Xingxing Wang just dropped some major truth bombs about the reality of robot labor at the HEIS meeting in Beijing Extreme mobility like the "Kung Fu" bot at the Gala is the prerequisite—stable movement is the foundation for robots to actually start working Unitree has been deploying humanoids at NIO and Geely since 2024 with a 100% success rate on single tasks like simple parts assembly The real challenge is long-sequence complex tasks with 20+ steps—success rates struggle here especially when handling tiny parts Whole-body teleoperation is the key technical breakthrough for capturing high-quality physical world data to solve the industry's data scarcity Standardization is now the priority—setting unified task metrics, evaluation systems, and safety baselines to move beyond one-off lab innovations In 2025 Unitree shipped over 5,500 humanoids with more than 6,500 units coming off the production line leading the global market in volume The end goal is clear—Unitree wants humanoids working in its own factories to manufacture even more robots, closing the automation loop Source: HEIS Annual Meeting #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #Unitree #SmartManufacturing #Automation

RoboHub🤖

31,755 Aufrufe • vor 4 Monaten

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

20,898 Aufrufe • vor 5 Monaten

Planning with the views: Can VLMs predict how each camera move changes the view, and plan many such moves ahead? We introduce ViewSuite with 6 DoF camera control and ~165K task instances, testing: Path-to-View View-to-Path Interactive View Planning A sharp Planning Gap emerges: + can roughly "track" how camera action changes views - cannot "compose" a plan towards a target view at all We then try to teach VLMs with Reinforcement Learning. - RL cannot teach VLMs such planning ability, only 2.5% success rate with Qwen2.5-VL-7B. + With View Graph Distillation (our RL-Graph-SFT framework), 2.5% → 47.8% Below, we answer these questions: Q1. What are the failure modes? Q2. How can we make RL work? Q3. What has the model learned? Can we open up the model to see before/after? Can such spatial priors transfer to other view related tasks? Led by Kangrui Wang, great to work with Linjie (Lindsey) Li Zhengyuan Yang Shiqi Chen Zihan "Zenus" Wang ✈️ ICML Fei-Fei Li Jiajun Wu Leonidas Guibas, Lijuan Wang. A joint efforts with Stanford AI Lab Stanford Vision and Learning Lab Microsoft Research.

Planning with the views: Can VLMs predict how each camera move changes the view, and plan many such moves ahead? We introduce ViewSuite with 6 DoF camera control and ~165K task instances, testing: Path-to-View View-to-Path Interactive View Planning A sharp Planning Gap emerges: + can roughly "track" how camera action changes views - cannot "compose" a plan towards a target view at all We then try to teach VLMs with Reinforcement Learning. - RL cannot teach VLMs such planning ability, only 2.5% success rate with Qwen2.5-VL-7B. + With View Graph Distillation (our RL-Graph-SFT framework), 2.5% → 47.8% Below, we answer these questions: Q1. What are the failure modes? Q2. How can we make RL work? Q3. What has the model learned? Can we open up the model to see before/after? Can such spatial priors transfer to other view related tasks? Led by Kangrui Wang, great to work with Linjie (Lindsey) Li Zhengyuan Yang Shiqi Chen Zihan "Zenus" Wang ✈️ ICML Fei-Fei Li Jiajun Wu Leonidas Guibas, Lijuan Wang. A joint efforts with Stanford AI Lab Stanford Vision and Learning Lab Microsoft Research.

Manling Li

59,301 Aufrufe • vor 29 Tagen

X-Humanoid just officially dropped Embodied Tien Kung 3.0, A universal platform designed to be way more open and developer-friendly. 🤖 Built on their Wise Kaiwu AI platform, this next-gen humanoid is all about slashing development costs. It’s a fully interoperable ecosystem that supports everything from tactile interaction to high-dynamic motion control at a full humanoid scale. ➤ Radical Openness: X-Humanoid is open-sourcing the full stack—robot body, motion control, VLM/VLA models, and the RoboMIND dataset. It fully supports ROS2, MQTT, and TCP/IP, so developers can customize use cases without re-engineering the basics. ➤ High-Performance Hardware: With high-torque integrated joints, Tien Kung 3.0 can clear 1-meter (3.3ft) obstacles and handle dexterous moves like kneeling and bending. It hits millimeter-level precision, making it a solid fit for industrial-grade tasks. ➤ True Autonomy: The bot runs a continuous perception-decision-execution loop. It uses world models to break down complex language commands and VLA models for real-time obstacle avoidance and navigation. ➤ Scalable Collaboration: The platform moves beyond single-unit tasks to support multi-robot collaboration with autonomous scheduling. It’s built to move embodied AI from the lab straight into real-world commercial and industrial environments. Source: X-Humanoid #Humanoid #OpenSource #Robotics #EmbodiedAI #PhysicalAI #Automation #XHumanoid #TienKung #WiseKaiwu

X-Humanoid just officially dropped Embodied Tien Kung 3.0, A universal platform designed to be way more open and developer-friendly. 🤖 Built on their Wise Kaiwu AI platform, this next-gen humanoid is all about slashing development costs. It’s a fully interoperable ecosystem that supports everything from tactile interaction to high-dynamic motion control at a full humanoid scale. ➤ Radical Openness: X-Humanoid is open-sourcing the full stack—robot body, motion control, VLM/VLA models, and the RoboMIND dataset. It fully supports ROS2, MQTT, and TCP/IP, so developers can customize use cases without re-engineering the basics. ➤ High-Performance Hardware: With high-torque integrated joints, Tien Kung 3.0 can clear 1-meter (3.3ft) obstacles and handle dexterous moves like kneeling and bending. It hits millimeter-level precision, making it a solid fit for industrial-grade tasks. ➤ True Autonomy: The bot runs a continuous perception-decision-execution loop. It uses world models to break down complex language commands and VLA models for real-time obstacle avoidance and navigation. ➤ Scalable Collaboration: The platform moves beyond single-unit tasks to support multi-robot collaboration with autonomous scheduling. It’s built to move embodied AI from the lab straight into real-world commercial and industrial environments. Source: X-Humanoid #Humanoid #OpenSource #Robotics #EmbodiedAI #PhysicalAI #Automation #XHumanoid #TienKung #WiseKaiwu

RoboHub🤖

49,440 Aufrufe • vor 5 Monaten

🚀Exciting News! We’re thrilled to introduce OpenWBT — the open-source, whole-body VR teleoperation system designed for humanoid across various robot types & virtual/physical worlds! 🤖✨ 💡Developed through collaboration between Galbot and Tsinghua University researchers! 🔥 Why it’s cool? ✅ Multi-robot support – Control different types of bots with one system! ✅ Cross virtual & real worlds – Operate bots seamlessly in simulator and the physical world. ✅ Built for YOU – Quick deployment with only a laptop and a VR headset, and create like never before! 💡 Calling all robot lovers & devs! Download, experiment, and help shape the future of robotics. Let’s #BuildTheFuture together! 🔗 Grab it now: GitHub： Welcome to our website： #OpenWBT #Robotics #OpenSource #Innovation #HumanoidRobots

🚀Exciting News! We’re thrilled to introduce OpenWBT — the open-source, whole-body VR teleoperation system designed for humanoid across various robot types & virtual/physical worlds! 🤖✨ 💡Developed through collaboration between Galbot and Tsinghua University researchers! 🔥 Why it’s cool? ✅ Multi-robot support – Control different types of bots with one system! ✅ Cross virtual & real worlds – Operate bots seamlessly in simulator and the physical world. ✅ Built for YOU – Quick deployment with only a laptop and a VR headset, and create like never before! 💡 Calling all robot lovers & devs! Download, experiment, and help shape the future of robotics. Let’s #BuildTheFuture together! 🔗 Grab it now: GitHub： Welcome to our website： #OpenWBT #Robotics #OpenSource #Innovation #HumanoidRobots

Galbot

38,356 Aufrufe • vor 1 Jahr

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

AK

41,736 Aufrufe • vor 1 Jahr

This work is not about a new technique. GMT (General Motion Tracking) shows good engineering practices that you can actually train a single unified whole-body control policy for all agile motion, and it works in the real world, directly with sim2real without adaptation. This is different from many existing works that trains/adapts one individual policy for each specific motion. Instead of just showing the beautiful motions/dances, this is actually practical. Because now you can convert all types of commands into real actions in real time without tuning anything. Visit to check out how we handle the following challenges: ✅ Partial Observability. ✅ Hardware Limitations. ✅ Unbalanced Data Distribution. ✅ Model Expressiveness. Work led by Zixuan Chen Mazeyu Ji

This work is not about a new technique. GMT (General Motion Tracking) shows good engineering practices that you can actually train a single unified whole-body control policy for all agile motion, and it works in the real world, directly with sim2real without adaptation. This is different from many existing works that trains/adapts one individual policy for each specific motion. Instead of just showing the beautiful motions/dances, this is actually practical. Because now you can convert all types of commands into real actions in real time without tuning anything. Visit to check out how we handle the following challenges: ✅ Partial Observability. ✅ Hardware Limitations. ✅ Unbalanced Data Distribution. ✅ Model Expressiveness. Work led by Zixuan Chen Mazeyu Ji

Xiaolong Wang

83,261 Aufrufe • vor 1 Jahr

Video: World’s first humanoid robot labor that swaps its own batteries to work endlessly | Jijo Malayil, Interesting Engineering Walker S2 uses dual-battery balancing and standardized modules to boost efficiency and ensure uninterrupted, optimized performance. In a leap for robotics, China’s UBTech has unveiled the Walker S2, the world’s first humanoid robot capable of fully autonomous battery swapping. Designed for non-stop industrial operations, the Walker S2 can replace its own power pack in just three minutes—no human intervention required. Equipped with advanced anthropomorphic bipedal locomotion and a hot-swappable battery system, Walker S2 is built to operate 24/7 across dynamic industrial environments. According to UBTech, the next-generation humanoid robot marks a major milestone in automation, bringing continuous, hands-free performance to the factory floor. In May 2025, UBTech Robotics and Huawei Technologies inked a significant partnership to accelerate the adoption of humanoid robots across China’s factories and households. Uninterrupted robot operations A video posted by the robotics firm opens with the sleek UBTech Walker S2 humanoid robot working in an industrial setting. The highlight, however, is its autonomous battery swap. Walker S2 approaches the charging station, carefully detaches its depleted power pack, and seamlessly installs a fresh one—all within about three minutes—without any human assistance, according to CGTN. The camera captures close-ups of the robot’s articulated limbs and the intelligent battery-handling mechanism, conveying precision and reliability. As the swap completes, Walker S2 resumes its duties, reinforcing the promise of uninterrupted, 24/7 operations in dynamic factory environments. UBTech’s Walker S2 humanoid robot is equipped with advanced dual-battery power balancing technology and uses standardized battery modules to optimize performance, reports CNEVPOST. This dual-battery system allows the robot to automatically switch to a backup battery in case of a main battery failure, ensuring that critical tasks are carried out without interruption. In addition to battery swapping, the robot can intelligently choose between charging and swapping based on task urgency, allowing it to manage energy dynamically and adapt to real-time operational demands. UBTech highlights these features as a step forward in deploying humanoid robots for industrial and domestic applications, combining flexibility, reliability, and autonomy in one intelligent platform. Factory intelligence upgrade Earlier in the year, UBTech unveiled a major advancement in humanoid robot collaboration, claiming the world’s first deployment of multiple humanoids working together across varied industrial tasks. Demonstrated at Zeekr’s 5G-enabled smart factory, the breakthrough centers on UBTech’s “BrainNet” framework, which orchestrates cooperative behavior through a cloud-device intelligence system. BrainNet integrates a “super brain” for high-level decision-making with an “intelligent sub-brain” for distributed multi-robot control. The super brain, powered by a proprietary large-scale multimodal reasoning model, handles complex production-line scheduling and decision-making. Meanwhile, the sub-brain coordinates real-time tasks using cross-field perception and Transformer-based control for dynamic adaptability. Together, they enable the Walker S1 humanoid robots to move beyond isolated operations and perform coordinated tasks with high precision and speed. The system is built on DeepSeek-R1 reasoning technology and trained on real-world data from automotive factory settings. Leveraging Retrieval-Augmented Generation (RAG), the model adapts to specific job functions and improves scalability across workstations. At Zeekr’s facility, dozens of Walker S1s now collaborate on tasks like assembly, inspection, and part handling. Using semantic VSLAM and shared mapping, they coordinate seamlessly via vision-based navigation and agile manipulation. UBTech says this marks a transition to “Practical Training 2.0,” where humanoid robots operate as a swarm, maximizing efficiency and setting the stage for next-generation intelligent manufacturing.

Video: World’s first humanoid robot labor that swaps its own batteries to work endlessly | Jijo Malayil, Interesting Engineering Walker S2 uses dual-battery balancing and standardized modules to boost efficiency and ensure uninterrupted, optimized performance. In a leap for robotics, China’s UBTech has unveiled the Walker S2, the world’s first humanoid robot capable of fully autonomous battery swapping. Designed for non-stop industrial operations, the Walker S2 can replace its own power pack in just three minutes—no human intervention required. Equipped with advanced anthropomorphic bipedal locomotion and a hot-swappable battery system, Walker S2 is built to operate 24/7 across dynamic industrial environments. According to UBTech, the next-generation humanoid robot marks a major milestone in automation, bringing continuous, hands-free performance to the factory floor. In May 2025, UBTech Robotics and Huawei Technologies inked a significant partnership to accelerate the adoption of humanoid robots across China’s factories and households. Uninterrupted robot operations A video posted by the robotics firm opens with the sleek UBTech Walker S2 humanoid robot working in an industrial setting. The highlight, however, is its autonomous battery swap. Walker S2 approaches the charging station, carefully detaches its depleted power pack, and seamlessly installs a fresh one—all within about three minutes—without any human assistance, according to CGTN. The camera captures close-ups of the robot’s articulated limbs and the intelligent battery-handling mechanism, conveying precision and reliability. As the swap completes, Walker S2 resumes its duties, reinforcing the promise of uninterrupted, 24/7 operations in dynamic factory environments. UBTech’s Walker S2 humanoid robot is equipped with advanced dual-battery power balancing technology and uses standardized battery modules to optimize performance, reports CNEVPOST. This dual-battery system allows the robot to automatically switch to a backup battery in case of a main battery failure, ensuring that critical tasks are carried out without interruption. In addition to battery swapping, the robot can intelligently choose between charging and swapping based on task urgency, allowing it to manage energy dynamically and adapt to real-time operational demands. UBTech highlights these features as a step forward in deploying humanoid robots for industrial and domestic applications, combining flexibility, reliability, and autonomy in one intelligent platform. Factory intelligence upgrade Earlier in the year, UBTech unveiled a major advancement in humanoid robot collaboration, claiming the world’s first deployment of multiple humanoids working together across varied industrial tasks. Demonstrated at Zeekr’s 5G-enabled smart factory, the breakthrough centers on UBTech’s “BrainNet” framework, which orchestrates cooperative behavior through a cloud-device intelligence system. BrainNet integrates a “super brain” for high-level decision-making with an “intelligent sub-brain” for distributed multi-robot control. The super brain, powered by a proprietary large-scale multimodal reasoning model, handles complex production-line scheduling and decision-making. Meanwhile, the sub-brain coordinates real-time tasks using cross-field perception and Transformer-based control for dynamic adaptability. Together, they enable the Walker S1 humanoid robots to move beyond isolated operations and perform coordinated tasks with high precision and speed. The system is built on DeepSeek-R1 reasoning technology and trained on real-world data from automotive factory settings. Leveraging Retrieval-Augmented Generation (RAG), the model adapts to specific job functions and improves scalability across workstations. At Zeekr’s facility, dozens of Walker S1s now collaborate on tasks like assembly, inspection, and part handling. Using semantic VSLAM and shared mapping, they coordinate seamlessly via vision-based navigation and agile manipulation. UBTech says this marks a transition to “Practical Training 2.0,” where humanoid robots operate as a swarm, maximizing efficiency and setting the stage for next-generation intelligent manufacturing.

Owen Gregorian

35,637 Aufrufe • vor 1 Jahr

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,080 Aufrufe • vor 4 Monaten

1X announces their latest reinforcement learning (RL) controller, which unlocks NEO's full-body mobility for home environments, enabling Redwood AI (1X's in-house AI model) to interact with the physical world more naturally and broadly. The unified controller supports walking in any direction, sitting, standing, kneeling, lying down, getting up, and climbing stairs using stereo RGB vision - critical for navigating real homes. The controller provides an “action interface” through which teleoperation or Redwood AI can interact in a safe, contact-rich manner with the physical world. Traditional walking controllers often rely on hand-crafted "shaping rewards" to produce human-like gaits, which are time-consuming and don’t scale well across tasks or movement directions. To address this, the team used motion capture data to guide the learning process. The RL system is trained to track kinematic reference trajectories derived from natural human motion while maintaining balance and rhythm in real time. To improve general-purpose utility, the controller goes beyond single-trajectory replay by introducing a two-part design: ⦿ A high-level kinematic planner generates smooth, human-like movement goals from simple input commands (e.g., joystick direction). ⦿ A low-level RL controller tracks these trajectories with dynamic stability. This structure enables smooth transitions between behaviors and resolves the challenge of controlling high-dimensional motion from coarse user input.

1X announces their latest reinforcement learning (RL) controller, which unlocks NEO's full-body mobility for home environments, enabling Redwood AI (1X's in-house AI model) to interact with the physical world more naturally and broadly. The unified controller supports walking in any direction, sitting, standing, kneeling, lying down, getting up, and climbing stairs using stereo RGB vision - critical for navigating real homes. The controller provides an “action interface” through which teleoperation or Redwood AI can interact in a safe, contact-rich manner with the physical world. Traditional walking controllers often rely on hand-crafted "shaping rewards" to produce human-like gaits, which are time-consuming and don’t scale well across tasks or movement directions. To address this, the team used motion capture data to guide the learning process. The RL system is trained to track kinematic reference trajectories derived from natural human motion while maintaining balance and rhythm in real time. To improve general-purpose utility, the controller goes beyond single-trajectory replay by introducing a two-part design: ⦿ A high-level kinematic planner generates smooth, human-like movement goals from simple input commands (e.g., joystick direction). ⦿ A low-level RL controller tracks these trajectories with dynamic stability. This structure enables smooth transitions between behaviors and resolves the challenge of controlling high-dimensional motion from coarse user input.

The Humanoid Hub

113,780 Aufrufe • vor 1 Jahr

Xiaomi Robotics Lab, alongside Tsinghua and HKUST, just figured out how to give humanoids a real "feel" for physics. 🤖🛹 Their new HAIC framework lets robots handle objects they can't even see by using high order proprioception (essentially digital muscle memory). Instead of relying on cameras, the bot analyzes the feedback from its own joints to infer exactly how an object is moving in its blind spot. Here is what this physical intuition actually looks like in practice: ➤ Blind Skateboarding: Hits a 100% success rate on gliding and dismounting without ever looking at the board. ➤ Heavy Loads: Pushes carts weighing up to 70kg (approx. 154 lbs) and pulls up to 20kg (approx. 44 lbs) with total stability. ➤ Obstructed Locomotion: Carries bulky boxes that block its view of the ground while flawlessly navigating stairs and slopes. ➤ Adaptive Brain: Proactively compensates for inertia and weight shifts by predicting velocity and acceleration on the fly. ➤ Generalization: Handles various box sizes and heavy weights without needing any specific retraining for the new dimensions. This is a huge deal for real world work where cameras are often blocked or lighting is too messy for pure vision. Paper: Project: #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #XiaomiRobotics #HAIC #SkateboardingRobot

Xiaomi Robotics Lab, alongside Tsinghua and HKUST, just figured out how to give humanoids a real "feel" for physics. 🤖🛹 Their new HAIC framework lets robots handle objects they can't even see by using high order proprioception (essentially digital muscle memory). Instead of relying on cameras, the bot analyzes the feedback from its own joints to infer exactly how an object is moving in its blind spot. Here is what this physical intuition actually looks like in practice: ➤ Blind Skateboarding: Hits a 100% success rate on gliding and dismounting without ever looking at the board. ➤ Heavy Loads: Pushes carts weighing up to 70kg (approx. 154 lbs) and pulls up to 20kg (approx. 44 lbs) with total stability. ➤ Obstructed Locomotion: Carries bulky boxes that block its view of the ground while flawlessly navigating stairs and slopes. ➤ Adaptive Brain: Proactively compensates for inertia and weight shifts by predicting velocity and acceleration on the fly. ➤ Generalization: Handles various box sizes and heavy weights without needing any specific retraining for the new dimensions. This is a huge deal for real world work where cameras are often blocked or lighting is too messy for pure vision. Paper: Project: #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #XiaomiRobotics #HAIC #SkateboardingRobot

RoboHub🤖

13,253 Aufrufe • vor 4 Monaten

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,533 Aufrufe • vor 6 Monaten

You might have seen the WuBOT performing at the 2026 Spring Festival Gala; however, most high-dynamic extreme motions you see are executed by overfitted tracking policies. Until now, training a unified policy capable of performing various extreme motions with a high success rate remained an unsolved challenge. We spent an entire year digging into the barrier between general tracking and extreme physical behaviors. After burning through dozens of G1 robots, we finally identified the bottleneck of learning and physical executability. With these discoveries, we developed OmniXtreme: the first general policy that can execute diverse extreme motions, including consecutive flips, extreme balancing, and even breakdancing with rapid contact switches! This capability is achieved by pre-training a flow-based generative control policy and then post-training with actuation-aware residual RL for complex physical dynamics—a step we found critical for successful real-world transfer. This work is a joint collaboration with Unitree. Together, we are pushing the physical limits of humanoid robots. It is incredibly exciting to see a general "robot gymnast" and "robot breakdancer" come to life! It was also our first time publishing a paper with XingXing, which was an enlightening experience. The model checkpoints are now released—we welcome you to play with them! 📦 📄 Paper: 🌐 Project: 💻 Code:

You might have seen the WuBOT performing at the 2026 Spring Festival Gala; however, most high-dynamic extreme motions you see are executed by overfitted tracking policies. Until now, training a unified policy capable of performing various extreme motions with a high success rate remained an unsolved challenge. We spent an entire year digging into the barrier between general tracking and extreme physical behaviors. After burning through dozens of G1 robots, we finally identified the bottleneck of learning and physical executability. With these discoveries, we developed OmniXtreme: the first general policy that can execute diverse extreme motions, including consecutive flips, extreme balancing, and even breakdancing with rapid contact switches! This capability is achieved by pre-training a flow-based generative control policy and then post-training with actuation-aware residual RL for complex physical dynamics—a step we found critical for successful real-world transfer. This work is a joint collaboration with Unitree. Together, we are pushing the physical limits of humanoid robots. It is incredibly exciting to see a general "robot gymnast" and "robot breakdancer" come to life! It was also our first time publishing a paper with XingXing, which was an enlightening experience. The model checkpoints are now released—we welcome you to play with them! 📦 📄 Paper: 🌐 Project: 💻 Code:

Siyuan Huang

106,645 Aufrufe • vor 4 Monaten