Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🤖 Ever dreamed of controlling a humanoid robot to perform complex, long-horizon tasks — using just a single Vision Pro? 🎉 Meet CLONE: a holistic, closed-loop, whole-body teleoperation system for long-horizon humanoid control! 🏃‍♂️🧍 CLONE enables rich and coordinated interactive tasks: 🥊 boxing 🏓 table tennis 🤲 object pickup... show more

Siyuan Huang

5,035 subscribers

66,737 views • 1 year ago •via X (Twitter)

Arts Science & Technology Education

Anya Rossi• Live Now

Private livecam show

4 Comments

Siyuan Huang1 year ago

Big shout out to all collaborators! This is the first project from the joint lab between BIGAI and UniTree Robotics — super excited about what’s coming next! 🚀

World Transformation Movement2 years ago

The holy grail of insight needed to rehabilitate the human race! Leading psychiatrist says “this is the most important interview of all time, it solves the human condition, ends suffering & turmoil at its source & finally makes sense of our lives!”

RobotSoul1 year ago

Awesome work!

Porters Reserve1 year ago

Novel concept but explain the use of having a small bipedal running around? get a kid and they will this independently.

Related Videos

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

Sawyer Merritt

624,770 views • 7 months ago

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi

Zhengyi “Zen” Luo

47,323 views • 2 years ago

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,399 views • 9 months ago

🦾 From seeing to doing. We're closing the loop between video prediction and real-world action. On the final day of Robbyant Open Source Week, we bring you LingBot-VA—the world's first causal video-action world model for generalist robot control. 🔥 Key Highlights: 🤖 Predicts & Acts: A single model generates both future video and the actions to make it real. 🧠 Remembers the Past: True long-term memory for complex, sequential tasks. ⚡ Learns in a Snap: Masters new skills with just 30-50 real-world demos. The result? 📈 SOTA on RoboTwin (92.9%) and LIBERO (98.5%), +20% over π0.5 on challenging real-world long-horizon & high-precision tasks. More below 👇 #AI #Robotics #EmbodiedAI #WorldModel #OpenSource

🦾 From seeing to doing. We're closing the loop between video prediction and real-world action. On the final day of Robbyant Open Source Week, we bring you LingBot-VA—the world's first causal video-action world model for generalist robot control. 🔥 Key Highlights: 🤖 Predicts & Acts: A single model generates both future video and the actions to make it real. 🧠 Remembers the Past: True long-term memory for complex, sequential tasks. ⚡ Learns in a Snap: Masters new skills with just 30-50 real-world demos. The result? 📈 SOTA on RoboTwin (92.9%) and LIBERO (98.5%), +20% over π0.5 on challenging real-world long-horizon & high-precision tasks. More below 👇 #AI #Robotics #EmbodiedAI #WorldModel #OpenSource

Robbyant

704,157 views • 7 months ago

We’ve raised $50 million in Series A funding from DST Global Partners, NVIDIA (NVIDIA’s venture capital arm), redalpine , prosusventures , and Moonfire Ventures , following our $7.35M seed round from Frst , Moonfire Ventures , and redalpine just a few months earlier, to build the autonomy stack that makes humanoid robots adaptive, intelligent, and ready for real-world deployment at scale. In less than a year, our team has shown that long-horizon whole-body humanoid control can scale across hardware and tasks by leveraging the power of simulation and reinforcement learning. This funding will help us grow our team, scale our compute and robot fleets, and accelerate the commercialization of our autonomy stack with OEM partners globally. You can find more details in the links shared in the comments.

We’ve raised $50 million in Series A funding from DST Global Partners, NVIDIA (NVIDIA’s venture capital arm), redalpine , prosusventures , and Moonfire Ventures , following our $7.35M seed round from Frst , Moonfire Ventures , and redalpine just a few months earlier, to build the autonomy stack that makes humanoid robots adaptive, intelligent, and ready for real-world deployment at scale. In less than a year, our team has shown that long-horizon whole-body humanoid control can scale across hardware and tasks by leveraging the power of simulation and reinforcement learning. This funding will help us grow our team, scale our compute and robot fleets, and accelerate the commercialization of our autonomy stack with OEM partners globally. You can find more details in the links shared in the comments.

Flexion

71,334 views • 9 months ago

X-Humanoid just officially dropped Embodied Tien Kung 3.0, A universal platform designed to be way more open and developer-friendly. 🤖 Built on their Wise Kaiwu AI platform, this next-gen humanoid is all about slashing development costs. It’s a fully interoperable ecosystem that supports everything from tactile interaction to high-dynamic motion control at a full humanoid scale. ➤ Radical Openness: X-Humanoid is open-sourcing the full stack—robot body, motion control, VLM/VLA models, and the RoboMIND dataset. It fully supports ROS2, MQTT, and TCP/IP, so developers can customize use cases without re-engineering the basics. ➤ High-Performance Hardware: With high-torque integrated joints, Tien Kung 3.0 can clear 1-meter (3.3ft) obstacles and handle dexterous moves like kneeling and bending. It hits millimeter-level precision, making it a solid fit for industrial-grade tasks. ➤ True Autonomy: The bot runs a continuous perception-decision-execution loop. It uses world models to break down complex language commands and VLA models for real-time obstacle avoidance and navigation. ➤ Scalable Collaboration: The platform moves beyond single-unit tasks to support multi-robot collaboration with autonomous scheduling. It’s built to move embodied AI from the lab straight into real-world commercial and industrial environments. Source: X-Humanoid #Humanoid #OpenSource #Robotics #EmbodiedAI #PhysicalAI #Automation #XHumanoid #TienKung #WiseKaiwu

X-Humanoid just officially dropped Embodied Tien Kung 3.0, A universal platform designed to be way more open and developer-friendly. 🤖 Built on their Wise Kaiwu AI platform, this next-gen humanoid is all about slashing development costs. It’s a fully interoperable ecosystem that supports everything from tactile interaction to high-dynamic motion control at a full humanoid scale. ➤ Radical Openness: X-Humanoid is open-sourcing the full stack—robot body, motion control, VLM/VLA models, and the RoboMIND dataset. It fully supports ROS2, MQTT, and TCP/IP, so developers can customize use cases without re-engineering the basics. ➤ High-Performance Hardware: With high-torque integrated joints, Tien Kung 3.0 can clear 1-meter (3.3ft) obstacles and handle dexterous moves like kneeling and bending. It hits millimeter-level precision, making it a solid fit for industrial-grade tasks. ➤ True Autonomy: The bot runs a continuous perception-decision-execution loop. It uses world models to break down complex language commands and VLA models for real-time obstacle avoidance and navigation. ➤ Scalable Collaboration: The platform moves beyond single-unit tasks to support multi-robot collaboration with autonomous scheduling. It’s built to move embodied AI from the lab straight into real-world commercial and industrial environments. Source: X-Humanoid #Humanoid #OpenSource #Robotics #EmbodiedAI #PhysicalAI #Automation #XHumanoid #TienKung #WiseKaiwu

RoboHub🤖

49,440 views • 6 months ago

Humanoid robots are cool. But most of them aren’t ready for real-world work. X-Humanoid just launched Embodied Tien Kung 3.0 - a full-size humanoid robot built to actually operate in industrial and commercial environments. What makes it different? • Strong, high-torque joints for heavy tasks • Can move dynamically and stay stable • Handles complex motions (even over obstacles) • Millimeter-level precision for detailed work It runs on the Wise KaiWu AI platform, which allows the robot to see, think, and act in a continuous loop with less human control and support for multi-robot collaboration. It also supports ROS2, MQTT, and TCP/IP, and offers more open hardware + software interfaces to make development easier for teams. The goal is simple: make humanoid robots practical, easier to build on, and ready for real-world use. Visit: Check out the official launch video below to see what it can do! 👇 #Robotics #HumanoidRobot #AI

Humanoid robots are cool. But most of them aren’t ready for real-world work. X-Humanoid just launched Embodied Tien Kung 3.0 - a full-size humanoid robot built to actually operate in industrial and commercial environments. What makes it different? • Strong, high-torque joints for heavy tasks • Can move dynamically and stay stable • Handles complex motions (even over obstacles) • Millimeter-level precision for detailed work It runs on the Wise KaiWu AI platform, which allows the robot to see, think, and act in a continuous loop with less human control and support for multi-robot collaboration. It also supports ROS2, MQTT, and TCP/IP, and offers more open hardware + software interfaces to make development easier for teams. The goal is simple: make humanoid robots practical, easier to build on, and ready for real-world use. Visit: Check out the official launch video below to see what it can do! 👇 #Robotics #HumanoidRobot #AI

Chidanand Tripathi

69,604 views • 6 months ago

NVIDIA MotionBricks brings humanoid robots closer to natural, fluid movement The system combines a modular latent generative model with Smart Primitives, packing more than 350,000 motion skills into a single backbone. It runs in real time at up to 15,000 FPS with just 2ms latency. Navigation, object interaction, sitting, jumping, obstacle crossing, and even different movement styles can be composed zero-shot, like reusable motion modules. It is already being used for whole-body control on the Unitree G1 and serves as the motion layer for NVIDIA GR00T Whole-Body Control. What makes this interesting is the move from training individual behaviors toward building a shared library of motion capabilities. If that approach scales, robot movement could become more modular, transferable, and easier to deploy across different tasks and environments.

NVIDIA MotionBricks brings humanoid robots closer to natural, fluid movement The system combines a modular latent generative model with Smart Primitives, packing more than 350,000 motion skills into a single backbone. It runs in real time at up to 15,000 FPS with just 2ms latency. Navigation, object interaction, sitting, jumping, obstacle crossing, and even different movement styles can be composed zero-shot, like reusable motion modules. It is already being used for whole-body control on the Unitree G1 and serves as the motion layer for NVIDIA GR00T Whole-Body Control. What makes this interesting is the move from training individual behaviors toward building a shared library of motion capabilities. If that approach scales, robot movement could become more modular, transferable, and easier to deploy across different tasks and environments.

CyberRobo

11,089 views • 2 months ago

Leju Robotics has launched the upgraded KUAVO 5/5-W humanoid and wheeled robots, designed to meet the rigorous demands of industrial automation. The robots feature a modular design that is key to flexibility: the legs are quickly replaceable with bipedal configurations (for complex terrain) or wheeled setups (for long-distance stability). The hands are also modular, supporting dexterous hands, grippers, or claws. This adaptability, combined with a rotatable and height-adjustable body, allows the robot to handle a vast range of factory tasks. Built for industrial stamina, the KUAVO series supports a 20kg maximum payload and an upper-body dual-arm payload of 10kg. It is engineered for continuous operation, boasting a battery life of over 8 hours. The control system achieves "silky-smooth" precision via doubled joint communication frequency. To drive high generalizability, Leju relies on a data-driven model. It established China's largest data acquisition center in Beijing (referred to as the "largest humanoid robot skill school") to collect over 6 million data points annually from 1:1 replications of real industrial scenes. This data feeds a universal multi-modal dataset, enabling the robot's "intelligent brain" to continuously learn complex tasks through closed-loop feedback in client factory environments.

Leju Robotics has launched the upgraded KUAVO 5/5-W humanoid and wheeled robots, designed to meet the rigorous demands of industrial automation. The robots feature a modular design that is key to flexibility: the legs are quickly replaceable with bipedal configurations (for complex terrain) or wheeled setups (for long-distance stability). The hands are also modular, supporting dexterous hands, grippers, or claws. This adaptability, combined with a rotatable and height-adjustable body, allows the robot to handle a vast range of factory tasks. Built for industrial stamina, the KUAVO series supports a 20kg maximum payload and an upper-body dual-arm payload of 10kg. It is engineered for continuous operation, boasting a battery life of over 8 hours. The control system achieves "silky-smooth" precision via doubled joint communication frequency. To drive high generalizability, Leju relies on a data-driven model. It established China's largest data acquisition center in Beijing (referred to as the "largest humanoid robot skill school") to collect over 6 million data points annually from 1:1 replications of real industrial scenes. This data feeds a universal multi-modal dataset, enabling the robot's "intelligent brain" to continuously learn complex tasks through closed-loop feedback in client factory environments.

RoboHub🤖

34,676 views • 10 months ago

China unveils humanoid robot with lifelike skin and blinking eyes built for daily life | Prabhat Ranjan Mishra, Interesting Engineering Large Language Models (LLMs) and Vision-Language Models (VLMs) help process and interpret complex data from human interactions. A Shanghai-based company has developed humanoid robots that appear as real as humans. The advanced bionic humanoid robot is integrated with self-supervised AI algorithms. Named Elf V1, the robot can perceive the world, communicate, learn, and interact intelligently with its surroundings. Developed by AheadForm Technology, the robot offers up to 30 degrees of freedom, powered by a precise control system and an advanced AI learning algorithm. Robot offers expressive facial features The robot offers expressive facial features, moving eyes, and synchronized speech. It can also convey emotions and understand human non-verbal cues, making interactions more natural and engaging. The robot has highly interactive capabilities and lifelike appearances. AheadForm expects that its robots could soon seamlessly integrate into daily life, providing assistance, companionship, and support across various industries. “We believe that by developing realistic and expressive robot heads, we can bridge the gap between humans and machines, fostering a new era of interactive and intelligent robotics,” said the company in a statement. Reports revealed that to avoid the “uncanny valley” effect and be able to interact with us, they are given lifelike skin and capabilities to read our emotions and respond appropriately using dynamic expression simulation and emotion generation tech. Bionic skin and high-precision control system The Elf V1 series of humanoids features 30 facial muscles animated by brushless micro-motors and managed by a high-precision control system. Paired with an ability to detect their users’ emotions with low latency and bionic skin, their facial expressions are nearly identical to those of humans, reported CGTN. The company claims it’s pioneering the development of realistic humanoid robots designed to revolutionize human-robot interaction. It’s enhancing sophisticated humanoid robot heads that can express emotions, perceive their environment, and interact seamlessly with humans. By combining cutting-edge AI and advanced robotics, AheadForm aims to bring life to machines and transform how humans engage with technology. AI models boost robots’ responsiveness Seamless integration of Large Language Models (LLMs) and Vision-Language Models (VLMs) into the humanoid robots can help them process and interpret complex data from human interactions, enabling the robot to learn and adapt in real-time, achieving human-level understanding and responsiveness. AheadForm uses Brushless Motors that deliver ultra-quiet operation and high responsiveness, specifically designed for precision facial movements in humanoid robots. With its compact size, lightweight design, and energy efficiency, this motor is the ideal choice for next-generation robots that require precise, subtle facial control to create a truly human-like experience. Previously, the company unveiled the Lan Series that features realistic humanoid robots with soft skin and 10 degrees of freedom, offering a lifelike appearance and intuitive movements. This series is designed for cost-efficiency, for applications prioritizing mobility and manipulation.

China unveils humanoid robot with lifelike skin and blinking eyes built for daily life | Prabhat Ranjan Mishra, Interesting Engineering Large Language Models (LLMs) and Vision-Language Models (VLMs) help process and interpret complex data from human interactions. A Shanghai-based company has developed humanoid robots that appear as real as humans. The advanced bionic humanoid robot is integrated with self-supervised AI algorithms. Named Elf V1, the robot can perceive the world, communicate, learn, and interact intelligently with its surroundings. Developed by AheadForm Technology, the robot offers up to 30 degrees of freedom, powered by a precise control system and an advanced AI learning algorithm. Robot offers expressive facial features The robot offers expressive facial features, moving eyes, and synchronized speech. It can also convey emotions and understand human non-verbal cues, making interactions more natural and engaging. The robot has highly interactive capabilities and lifelike appearances. AheadForm expects that its robots could soon seamlessly integrate into daily life, providing assistance, companionship, and support across various industries. “We believe that by developing realistic and expressive robot heads, we can bridge the gap between humans and machines, fostering a new era of interactive and intelligent robotics,” said the company in a statement. Reports revealed that to avoid the “uncanny valley” effect and be able to interact with us, they are given lifelike skin and capabilities to read our emotions and respond appropriately using dynamic expression simulation and emotion generation tech. Bionic skin and high-precision control system The Elf V1 series of humanoids features 30 facial muscles animated by brushless micro-motors and managed by a high-precision control system. Paired with an ability to detect their users’ emotions with low latency and bionic skin, their facial expressions are nearly identical to those of humans, reported CGTN. The company claims it’s pioneering the development of realistic humanoid robots designed to revolutionize human-robot interaction. It’s enhancing sophisticated humanoid robot heads that can express emotions, perceive their environment, and interact seamlessly with humans. By combining cutting-edge AI and advanced robotics, AheadForm aims to bring life to machines and transform how humans engage with technology. AI models boost robots’ responsiveness Seamless integration of Large Language Models (LLMs) and Vision-Language Models (VLMs) into the humanoid robots can help them process and interpret complex data from human interactions, enabling the robot to learn and adapt in real-time, achieving human-level understanding and responsiveness. AheadForm uses Brushless Motors that deliver ultra-quiet operation and high responsiveness, specifically designed for precision facial movements in humanoid robots. With its compact size, lightweight design, and energy efficiency, this motor is the ideal choice for next-generation robots that require precise, subtle facial control to create a truly human-like experience. Previously, the company unveiled the Lan Series that features realistic humanoid robots with soft skin and 10 degrees of freedom, offering a lifelike appearance and intuitive movements. This series is designed for cost-efficiency, for applications prioritizing mobility and manipulation.

Owen Gregorian

179,005 views • 10 months ago

This is how ALOHA's "teleoperation" system works - a fancy word for "remote control". Training robots will be more and more like playing games in the physical world. A human operates a "joystick++" to perform tasks and collect data, or intervene if there's any safety concern. There's actually a learning curve to master the controller, much like practicing gaming skills. Teleoperation can be done in many different ways. ALOHA is an impressive custom-built system with very low cost. Here're a few alternatives: (1) Motion Capture (MoCap): apply the MoCap systems used for Hollywood movies to capture the fine-grained motions of hand joints. There would be no "embodiment gap" if the robot hand has 5 fingers. For instance, a demonstrator can wear a CyberGlove ( and manipulate the objects. CyberGlove will capture the motion signals & haptic feedback in real-time, which can be re-targeted onto the humanoid. (2) Wearing gloves & markers can be clumsy. An alternative way to do MoCap is through computer vision. DexPilot from NVIDIA enables marker-less and glove-free data collection. The human operator simply uses their bare hands to perform the tasks. 4 Intel RealSense depth cameras and 2 NVIDIA Titan XP GPUs (yeah, 2019 work) translate the pixels to precise motion signals for robot learning. (3) VR Headset: turn the training room into a VR game and "role play" the robot. This has the advantage of scalable remote data collection - annotators from around the world can contribute without coming onsite. VR demonstration technique appeared in research projects like the iGibson home robot simulator, an initiative that I participated in at Stanford: Behind-the-scene video by Litian Liang

This is how ALOHA's "teleoperation" system works - a fancy word for "remote control". Training robots will be more and more like playing games in the physical world. A human operates a "joystick++" to perform tasks and collect data, or intervene if there's any safety concern. There's actually a learning curve to master the controller, much like practicing gaming skills. Teleoperation can be done in many different ways. ALOHA is an impressive custom-built system with very low cost. Here're a few alternatives: (1) Motion Capture (MoCap): apply the MoCap systems used for Hollywood movies to capture the fine-grained motions of hand joints. There would be no "embodiment gap" if the robot hand has 5 fingers. For instance, a demonstrator can wear a CyberGlove ( and manipulate the objects. CyberGlove will capture the motion signals & haptic feedback in real-time, which can be re-targeted onto the humanoid. (2) Wearing gloves & markers can be clumsy. An alternative way to do MoCap is through computer vision. DexPilot from NVIDIA enables marker-less and glove-free data collection. The human operator simply uses their bare hands to perform the tasks. 4 Intel RealSense depth cameras and 2 NVIDIA Titan XP GPUs (yeah, 2019 work) translate the pixels to precise motion signals for robot learning. (3) VR Headset: turn the training room into a VR game and "role play" the robot. This has the advantage of scalable remote data collection - annotators from around the world can contribute without coming onsite. VR demonstration technique appeared in research projects like the iGibson home robot simulator, an initiative that I participated in at Stanford: Behind-the-scene video by Litian Liang

Jim Fan

124,588 views • 2 years ago

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 views • 4 months ago

There’s not enough hype about Flexion Robotics. They are literally one of the sexiest European companies these days! 🇪🇺 I had a chance to meet the founder Nikita last year and learn his story. He turned down offers from top physical AI companies to build his own. The reason was his specific vision of how robot intelligence should be built to actually scale. So he decided to build one himself. Flexion is building an ultimate brain for humanoid robots. Yesterday they released Reflect v1.0. It’s an intelligence platform for long-horizon autonomous missions. The system has three layers. At the top, a custom vision-language model acts as mission control, observing the robot's camera feed and replanning continuously. Below that, a motion layer combines a VLA trained on real-world data with reinforcement learning-based skills. It translates perception into navigation, scene interaction, and manipulation commands. At the bottom, a whole-body controller runs in real-time, maintaining balance while the robot navigates and performs tasks. I highly recommend them visiting Zurich! 🇨🇭 Nikita Rudin cheapeu bas on the launch! 🎩 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://

There’s not enough hype about Flexion Robotics. They are literally one of the sexiest European companies these days! 🇪🇺 I had a chance to meet the founder Nikita last year and learn his story. He turned down offers from top physical AI companies to build his own. The reason was his specific vision of how robot intelligence should be built to actually scale. So he decided to build one himself. Flexion is building an ultimate brain for humanoid robots. Yesterday they released Reflect v1.0. It’s an intelligence platform for long-horizon autonomous missions. The system has three layers. At the top, a custom vision-language model acts as mission control, observing the robot's camera feed and replanning continuously. Below that, a motion layer combines a VLA trained on real-world data with reinforcement learning-based skills. It translates perception into navigation, scene interaction, and manipulation commands. At the bottom, a whole-body controller runs in real-time, maintaining balance while the robot navigates and performs tasks. I highly recommend them visiting Zurich! 🇨🇭 Nikita Rudin cheapeu bas on the launch! 🎩 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → http://

Lukas Ziegler

23,161 views • 1 month ago

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

466,610 views • 1 year ago

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the motion of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,565 views • 2 years ago

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Rohan Paul

11,253 views • 1 month ago

We believe we’re the first robotics company to demonstrate a robot peeling an apple with dual dexterous human-like hands. This breakthrough closes a key gap in robotics, achieving bimanual, contact-rich manipulation and moving far beyond the limits of simple grippers. 🧵↓ Today’s AI models (VLMs) are excellent at perception but struggle with action. Controlling high-degree-of-freedom hands for tasks like this is incredibly complex, and precise finger-level teleoperation is nearly impossible for humans. Our first step was a shared-autonomy system: rather than controlling every finger, the operator triggers pre-learned skills like a “rotate apple or tennis ball” primitive via a keyboard press or pedal. This makes scalable data collection and RL training possible. How does the AI manage this? We created "MoDE-VLA" (Mixture of Dexterous Experts). It fuses vision, language, force, and touch data by using a team of specialist "experts," making control in high-dimensional spaces stable and effective. The combination of these two innovations allows for seamless, contact-rich manipulation. The human provides high-level guidance, and the robot executes the complex in-hand coordination required. This work paves the way for robots that can safely handle delicate tasks in human environments. Want the full technical details? 📄 Read the full research paper: Visit us at NVIDIA GTC Booth #1838, Hall 3 to learn more! #Robotics #AI #DexterousManipulation #VLA #NVIDIAGTC Nancy Villicaña NVIDIA GTC

We believe we’re the first robotics company to demonstrate a robot peeling an apple with dual dexterous human-like hands. This breakthrough closes a key gap in robotics, achieving bimanual, contact-rich manipulation and moving far beyond the limits of simple grippers. 🧵↓ Today’s AI models (VLMs) are excellent at perception but struggle with action. Controlling high-degree-of-freedom hands for tasks like this is incredibly complex, and precise finger-level teleoperation is nearly impossible for humans. Our first step was a shared-autonomy system: rather than controlling every finger, the operator triggers pre-learned skills like a “rotate apple or tennis ball” primitive via a keyboard press or pedal. This makes scalable data collection and RL training possible. How does the AI manage this? We created "MoDE-VLA" (Mixture of Dexterous Experts). It fuses vision, language, force, and touch data by using a team of specialist "experts," making control in high-dimensional spaces stable and effective. The combination of these two innovations allows for seamless, contact-rich manipulation. The human provides high-level guidance, and the robot executes the complex in-hand coordination required. This work paves the way for robots that can safely handle delicate tasks in human environments. Want the full technical details? 📄 Read the full research paper: Visit us at NVIDIA GTC Booth #1838, Hall 3 to learn more! #Robotics #AI #DexterousManipulation #VLA #NVIDIAGTC Nancy Villicaña NVIDIA GTC

Sharpa

20,429 views • 5 months ago