Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

Wenli Xiao

3,159 subscribers

70,449 görüntüleme • 2 yıl önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

10 Yorum

Wenli Xiao profil fotoğrafı

Wenli Xiao2 yıl önce

Please checkout our website for more videos! Website: Paper Link: Video: Team: @ChongZitaZhang @_wenlixiao @TairanHe99 @GuanyaShi Thanks for the help from: @arthurallshire @JasonJZLiu @MiladShafieeA @guanqi_he Justin Macey and Jessica Hodgins

Chaoyi Pan profil fotoğrafı

Chaoyi Pan2 yıl önce

Congrats @_wenlixiao!

Wenli Xiao profil fotoğrafı

Wenli Xiao2 yıl önce

Thank you Chaoyi!

Haoru Xue profil fotoğrafı

Haoru Xue2 yıl önce

Congratulations! @_wenlixiao

Wenli Xiao profil fotoğrafı

Wenli Xiao2 yıl önce

Thank you Haoru!

Milad Shafiee profil fotoğrafı

Milad Shafiee2 yıl önce

Nice to see doggy learned to play😁 nice work 🙌

Shiqi Yang profil fotoğrafı

Shiqi Yang2 yıl önce

Congrats!

Wenli Xiao profil fotoğrafı

Wenli Xiao2 yıl önce

Thanks Shiqi, your 📺Open-Television is also very impressive, we should collaborate someday🦾

Chen Tessler profil fotoğrafı

Chen Tessler2 yıl önce

Awesome!

Mikhail Kakanov profil fotoğrafı

Mikhail Kakanov2 yıl önce

Looks really cool 👍

Benzer Videolar

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

The Humanoid Hub

19,563 görüntüleme • 2 ay önce

X-Humanoid just dropped TG-VLA — the first full-size, whole-body VLA framework for humanoids. 🤖 Most VLA demos today still look arm-centric: see, plan, reach, grasp. The legs are mostly there to carry the arms. TG-VLA pushes action into the whole body: → HEX: task context & cross-embodiment learning → HAF-VLA: high-DoF motion → structured action flows → DSRL-DCT: online RL through a compressed latent space The real shift isn't a smarter hand — it's locomotion, torso control, balance, and manipulation inside one control loop. That's the move: from mobile dual-arm machines to full-body agents.

X-Humanoid just dropped TG-VLA — the first full-size, whole-body VLA framework for humanoids. 🤖 Most VLA demos today still look arm-centric: see, plan, reach, grasp. The legs are mostly there to carry the arms. TG-VLA pushes action into the whole body: → HEX: task context & cross-embodiment learning → HAF-VLA: high-DoF motion → structured action flows → DSRL-DCT: online RL through a compressed latent space The real shift isn't a smarter hand — it's locomotion, torso control, balance, and manipulation inside one control loop. That's the move: from mobile dual-arm machines to full-body agents.

RoboHub🤖

42,167 görüntüleme • 2 ay önce

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

Ashish Kapoor

118,467 görüntüleme • 1 yıl önce

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi

Zhengyi “Zen” Luo

47,345 görüntüleme • 2 yıl önce

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 görüntüleme • 1 yıl önce

NVIDIA MotionBricks brings humanoid robots closer to natural, fluid movement The system combines a modular latent generative model with Smart Primitives, packing more than 350,000 motion skills into a single backbone. It runs in real time at up to 15,000 FPS with just 2ms latency. Navigation, object interaction, sitting, jumping, obstacle crossing, and even different movement styles can be composed zero-shot, like reusable motion modules. It is already being used for whole-body control on the Unitree G1 and serves as the motion layer for NVIDIA GR00T Whole-Body Control. What makes this interesting is the move from training individual behaviors toward building a shared library of motion capabilities. If that approach scales, robot movement could become more modular, transferable, and easier to deploy across different tasks and environments.

NVIDIA MotionBricks brings humanoid robots closer to natural, fluid movement The system combines a modular latent generative model with Smart Primitives, packing more than 350,000 motion skills into a single backbone. It runs in real time at up to 15,000 FPS with just 2ms latency. Navigation, object interaction, sitting, jumping, obstacle crossing, and even different movement styles can be composed zero-shot, like reusable motion modules. It is already being used for whole-body control on the Unitree G1 and serves as the motion layer for NVIDIA GR00T Whole-Body Control. What makes this interesting is the move from training individual behaviors toward building a shared library of motion capabilities. If that approach scales, robot movement could become more modular, transferable, and easier to deploy across different tasks and environments.

CyberRobo

11,089 görüntüleme • 3 ay önce

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

RoboPapers

44,825 görüntüleme • 9 ay önce

Excited to introduce SOLE-R1, a video-language reasoning model for zero-shot reward prediction for robot manipulation tasks! SOLE-R1 reasoning can serve as the SOLE signal for learning new tasks (completely from scratch) through online RL - i.e., robots start with random actions and learn previously unseen tasks guided only by SOLE-R1 rewards, without any demonstrations, ground-truth rewards, success indicators, or task-specific tuning. SOLE-R1 significantly outperforms strong baselines (e.g., Robometer, RoboReward, TOPReward, GPT-5, Gemini-3-Pro) in zero-shot online RL when evaluated across 40 tasks - including a real-world tabletop manipulation setting and 4 sim environments (LIBERO, ManiSkill, Meta-World, RoboSuite). We open source all models, training data, and code. Website, demos, and paper at: 🧵 (1/6)

Excited to introduce SOLE-R1, a video-language reasoning model for zero-shot reward prediction for robot manipulation tasks! SOLE-R1 reasoning can serve as the SOLE signal for learning new tasks (completely from scratch) through online RL - i.e., robots start with random actions and learn previously unseen tasks guided only by SOLE-R1 rewards, without any demonstrations, ground-truth rewards, success indicators, or task-specific tuning. SOLE-R1 significantly outperforms strong baselines (e.g., Robometer, RoboReward, TOPReward, GPT-5, Gemini-3-Pro) in zero-shot online RL when evaluated across 40 tasks - including a real-world tabletop manipulation setting and 4 sim environments (LIBERO, ManiSkill, Meta-World, RoboSuite). We open source all models, training data, and code. Website, demos, and paper at: 🧵 (1/6)

Philip Schroeder

19,408 görüntüleme • 3 ay önce

LimX Dynamics just released LimX Luna, officially described by the company as the first mass-deliverable full-size female humanoid robot. 🤖 This is not aimed at warehouses; it is built for malls, theme parks, auto shows, brand events, and other commercial spaces where attention is the product. Luna stands 160 cm tall, has 27 body DoF excluding end effectors, and combines whole-body motion control with voice, vision, facial expressions, gesture interaction, and motion imitation. The sharper part is the content stack: users can upload dance videos, build scenes with zero-code tools in LimX Studio, and run synchronized shows with up to 200 robots. That makes Luna less like a one-off demo machine and more like a programmable entertainment platform for foot traffic, performances, and social clips. For humanoids, this is a useful shift: not only “can it do labor,” but “can it create a consumer experience people actually stop to watch.”

RoboHub🤖

14,205 görüntüleme • 4 ay önce

How humanoids move from "Kung Fu" mode to "Blue-collar" work 🤖🦾 Unitree founder Xingxing Wang just dropped some major truth bombs about the reality of robot labor at the HEIS meeting in Beijing Extreme mobility like the "Kung Fu" bot at the Gala is the prerequisite—stable movement is the foundation for robots to actually start working Unitree has been deploying humanoids at NIO and Geely since 2024 with a 100% success rate on single tasks like simple parts assembly The real challenge is long-sequence complex tasks with 20+ steps—success rates struggle here especially when handling tiny parts Whole-body teleoperation is the key technical breakthrough for capturing high-quality physical world data to solve the industry's data scarcity Standardization is now the priority—setting unified task metrics, evaluation systems, and safety baselines to move beyond one-off lab innovations In 2025 Unitree shipped over 5,500 humanoids with more than 6,500 units coming off the production line leading the global market in volume The end goal is clear—Unitree wants humanoids working in its own factories to manufacture even more robots, closing the automation loop Source: HEIS Annual Meeting #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #Unitree #SmartManufacturing #Automation

How humanoids move from "Kung Fu" mode to "Blue-collar" work 🤖🦾 Unitree founder Xingxing Wang just dropped some major truth bombs about the reality of robot labor at the HEIS meeting in Beijing Extreme mobility like the "Kung Fu" bot at the Gala is the prerequisite—stable movement is the foundation for robots to actually start working Unitree has been deploying humanoids at NIO and Geely since 2024 with a 100% success rate on single tasks like simple parts assembly The real challenge is long-sequence complex tasks with 20+ steps—success rates struggle here especially when handling tiny parts Whole-body teleoperation is the key technical breakthrough for capturing high-quality physical world data to solve the industry's data scarcity Standardization is now the priority—setting unified task metrics, evaluation systems, and safety baselines to move beyond one-off lab innovations In 2025 Unitree shipped over 5,500 humanoids with more than 6,500 units coming off the production line leading the global market in volume The end goal is clear—Unitree wants humanoids working in its own factories to manufacture even more robots, closing the automation loop Source: HEIS Annual Meeting #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #Unitree #SmartManufacturing #Automation

RoboHub🤖

31,755 görüntüleme • 7 ay önce

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

20,898 görüntüleme • 7 ay önce

Planning with the views: Can VLMs predict how each camera move changes the view, and plan many such moves ahead? We introduce ViewSuite with 6 DoF camera control and ~165K task instances, testing: Path-to-View View-to-Path Interactive View Planning A sharp Planning Gap emerges: + can roughly "track" how camera action changes views - cannot "compose" a plan towards a target view at all We then try to teach VLMs with Reinforcement Learning. - RL cannot teach VLMs such planning ability, only 2.5% success rate with Qwen2.5-VL-7B. + With View Graph Distillation (our RL-Graph-SFT framework), 2.5% → 47.8% Below, we answer these questions: Q1. What are the failure modes? Q2. How can we make RL work? Q3. What has the model learned? Can we open up the model to see before/after? Can such spatial priors transfer to other view related tasks? Led by Kangrui Wang, great to work with Linjie (Lindsey) Li Zhengyuan Yang Shiqi Chen Zihan "Zenus" Wang Fei-Fei Li Jiajun Wu Leonidas Guibas, Lijuan Wang. A joint efforts with Stanford AI Lab Stanford Vision and Learning Lab Microsoft Research.

Planning with the views: Can VLMs predict how each camera move changes the view, and plan many such moves ahead? We introduce ViewSuite with 6 DoF camera control and ~165K task instances, testing: Path-to-View View-to-Path Interactive View Planning A sharp Planning Gap emerges: + can roughly "track" how camera action changes views - cannot "compose" a plan towards a target view at all We then try to teach VLMs with Reinforcement Learning. - RL cannot teach VLMs such planning ability, only 2.5% success rate with Qwen2.5-VL-7B. + With View Graph Distillation (our RL-Graph-SFT framework), 2.5% → 47.8% Below, we answer these questions: Q1. What are the failure modes? Q2. How can we make RL work? Q3. What has the model learned? Can we open up the model to see before/after? Can such spatial priors transfer to other view related tasks? Led by Kangrui Wang, great to work with Linjie (Lindsey) Li Zhengyuan Yang Shiqi Chen Zihan "Zenus" Wang Fei-Fei Li Jiajun Wu Leonidas Guibas, Lijuan Wang. A joint efforts with Stanford AI Lab Stanford Vision and Learning Lab Microsoft Research.

Manling Li

63,397 görüntüleme • 3 ay önce

X-Humanoid just officially dropped Embodied Tien Kung 3.0, A universal platform designed to be way more open and developer-friendly. 🤖 Built on their Wise Kaiwu AI platform, this next-gen humanoid is all about slashing development costs. It’s a fully interoperable ecosystem that supports everything from tactile interaction to high-dynamic motion control at a full humanoid scale. ➤ Radical Openness: X-Humanoid is open-sourcing the full stack—robot body, motion control, VLM/VLA models, and the RoboMIND dataset. It fully supports ROS2, MQTT, and TCP/IP, so developers can customize use cases without re-engineering the basics. ➤ High-Performance Hardware: With high-torque integrated joints, Tien Kung 3.0 can clear 1-meter (3.3ft) obstacles and handle dexterous moves like kneeling and bending. It hits millimeter-level precision, making it a solid fit for industrial-grade tasks. ➤ True Autonomy: The bot runs a continuous perception-decision-execution loop. It uses world models to break down complex language commands and VLA models for real-time obstacle avoidance and navigation. ➤ Scalable Collaboration: The platform moves beyond single-unit tasks to support multi-robot collaboration with autonomous scheduling. It’s built to move embodied AI from the lab straight into real-world commercial and industrial environments. Source: X-Humanoid #Humanoid #OpenSource #Robotics #EmbodiedAI #PhysicalAI #Automation #XHumanoid #TienKung #WiseKaiwu

X-Humanoid just officially dropped Embodied Tien Kung 3.0, A universal platform designed to be way more open and developer-friendly. 🤖 Built on their Wise Kaiwu AI platform, this next-gen humanoid is all about slashing development costs. It’s a fully interoperable ecosystem that supports everything from tactile interaction to high-dynamic motion control at a full humanoid scale. ➤ Radical Openness: X-Humanoid is open-sourcing the full stack—robot body, motion control, VLM/VLA models, and the RoboMIND dataset. It fully supports ROS2, MQTT, and TCP/IP, so developers can customize use cases without re-engineering the basics. ➤ High-Performance Hardware: With high-torque integrated joints, Tien Kung 3.0 can clear 1-meter (3.3ft) obstacles and handle dexterous moves like kneeling and bending. It hits millimeter-level precision, making it a solid fit for industrial-grade tasks. ➤ True Autonomy: The bot runs a continuous perception-decision-execution loop. It uses world models to break down complex language commands and VLA models for real-time obstacle avoidance and navigation. ➤ Scalable Collaboration: The platform moves beyond single-unit tasks to support multi-robot collaboration with autonomous scheduling. It’s built to move embodied AI from the lab straight into real-world commercial and industrial environments. Source: X-Humanoid #Humanoid #OpenSource #Robotics #EmbodiedAI #PhysicalAI #Automation #XHumanoid #TienKung #WiseKaiwu

RoboHub🤖

49,440 görüntüleme • 7 ay önce

🚀Exciting News! We’re thrilled to introduce OpenWBT — the open-source, whole-body VR teleoperation system designed for humanoid across various robot types & virtual/physical worlds! 🤖✨ 💡Developed through collaboration between Galbot and Tsinghua University researchers! 🔥 Why it’s cool? ✅ Multi-robot support – Control different types of bots with one system! ✅ Cross virtual & real worlds – Operate bots seamlessly in simulator and the physical world. ✅ Built for YOU – Quick deployment with only a laptop and a VR headset, and create like never before! 💡 Calling all robot lovers & devs! Download, experiment, and help shape the future of robotics. Let’s #BuildTheFuture together! 🔗 Grab it now: GitHub： Welcome to our website： #OpenWBT #Robotics #OpenSource #Innovation #HumanoidRobots

🚀Exciting News! We’re thrilled to introduce OpenWBT — the open-source, whole-body VR teleoperation system designed for humanoid across various robot types & virtual/physical worlds! 🤖✨ 💡Developed through collaboration between Galbot and Tsinghua University researchers! 🔥 Why it’s cool? ✅ Multi-robot support – Control different types of bots with one system! ✅ Cross virtual & real worlds – Operate bots seamlessly in simulator and the physical world. ✅ Built for YOU – Quick deployment with only a laptop and a VR headset, and create like never before! 💡 Calling all robot lovers & devs! Download, experiment, and help shape the future of robotics. Let’s #BuildTheFuture together! 🔗 Grab it now: GitHub： Welcome to our website： #OpenWBT #Robotics #OpenSource #Innovation #HumanoidRobots

Galbot

38,650 görüntüleme • 1 yıl önce

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

AK

41,747 görüntüleme • 2 yıl önce

This work is not about a new technique. GMT (General Motion Tracking) shows good engineering practices that you can actually train a single unified whole-body control policy for all agile motion, and it works in the real world, directly with sim2real without adaptation. This is different from many existing works that trains/adapts one individual policy for each specific motion. Instead of just showing the beautiful motions/dances, this is actually practical. Because now you can convert all types of commands into real actions in real time without tuning anything. Visit to check out how we handle the following challenges: ✅ Partial Observability. ✅ Hardware Limitations. ✅ Unbalanced Data Distribution. ✅ Model Expressiveness. Work led by Zixuan Chen Mazeyu Ji

This work is not about a new technique. GMT (General Motion Tracking) shows good engineering practices that you can actually train a single unified whole-body control policy for all agile motion, and it works in the real world, directly with sim2real without adaptation. This is different from many existing works that trains/adapts one individual policy for each specific motion. Instead of just showing the beautiful motions/dances, this is actually practical. Because now you can convert all types of commands into real actions in real time without tuning anything. Visit to check out how we handle the following challenges: ✅ Partial Observability. ✅ Hardware Limitations. ✅ Unbalanced Data Distribution. ✅ Model Expressiveness. Work led by Zixuan Chen Mazeyu Ji

Xiaolong Wang

83,347 görüntüleme • 1 yıl önce

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Not a preplanned motion sequence. A robot deciding mid-jump what to do next. [📍 paper + demo] Researchers just showed a humanoid doing real parkour using only onboard perception. No motion script, no fixed obstacle layout. The system is called Perceptive Humanoid Parkour (PHP). Instead of memorizing a path, the robot reads depth from its cameras and continuously chooses actions. Step, vault, climb, or roll depending on what geometry appears in front of it. To make that possible, they combine three ideas: First, they stitch together human motion clips into long movement references so the robot learns fluid transitions instead of isolated tricks. Second, they train tracking policies with reinforcement learning so contacts land at the right time and the robot keeps balance during dynamic moves. Finally, everything is distilled into one perception policy that runs directly from depth input to action selection. The result on a Unitree G1: about 3 m/s vaults wall climbs up to 1.25 m nearly one minute continuous obstacle traversal adapting when obstacles move What matters is not the tricks. It is the shift in capability. Earlier humanoids executed motions. This one navigates situations. Once robots react to geometry instead of replaying trajectories, environments stop needing to be predictable. Warehouses, homes, and outdoors suddenly become the same problem. Thanks for sharing, Zhen Wu! Paper + demo: ——— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,127 görüntüleme • 7 ay önce

Xiaomi Robotics Lab, alongside Tsinghua and HKUST, just figured out how to give humanoids a real "feel" for physics. 🤖🛹 Their new HAIC framework lets robots handle objects they can't even see by using high order proprioception (essentially digital muscle memory). Instead of relying on cameras, the bot analyzes the feedback from its own joints to infer exactly how an object is moving in its blind spot. Here is what this physical intuition actually looks like in practice: ➤ Blind Skateboarding: Hits a 100% success rate on gliding and dismounting without ever looking at the board. ➤ Heavy Loads: Pushes carts weighing up to 70kg (approx. 154 lbs) and pulls up to 20kg (approx. 44 lbs) with total stability. ➤ Obstructed Locomotion: Carries bulky boxes that block its view of the ground while flawlessly navigating stairs and slopes. ➤ Adaptive Brain: Proactively compensates for inertia and weight shifts by predicting velocity and acceleration on the fly. ➤ Generalization: Handles various box sizes and heavy weights without needing any specific retraining for the new dimensions. This is a huge deal for real world work where cameras are often blocked or lighting is too messy for pure vision. Paper: Project: #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #XiaomiRobotics #HAIC #SkateboardingRobot

Xiaomi Robotics Lab, alongside Tsinghua and HKUST, just figured out how to give humanoids a real "feel" for physics. 🤖🛹 Their new HAIC framework lets robots handle objects they can't even see by using high order proprioception (essentially digital muscle memory). Instead of relying on cameras, the bot analyzes the feedback from its own joints to infer exactly how an object is moving in its blind spot. Here is what this physical intuition actually looks like in practice: ➤ Blind Skateboarding: Hits a 100% success rate on gliding and dismounting without ever looking at the board. ➤ Heavy Loads: Pushes carts weighing up to 70kg (approx. 154 lbs) and pulls up to 20kg (approx. 44 lbs) with total stability. ➤ Obstructed Locomotion: Carries bulky boxes that block its view of the ground while flawlessly navigating stairs and slopes. ➤ Adaptive Brain: Proactively compensates for inertia and weight shifts by predicting velocity and acceleration on the fly. ➤ Generalization: Handles various box sizes and heavy weights without needing any specific retraining for the new dimensions. This is a huge deal for real world work where cameras are often blocked or lighting is too messy for pure vision. Paper: Project: #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #XiaomiRobotics #HAIC #SkateboardingRobot

RoboHub🤖

13,265 görüntüleme • 6 ay önce