Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Excited to introduce OmniClone, a robust teleoperation system for humanoid mobile manipulation. While systems like TWIST2 and SONIC paved the way, we put efforts into solving the critical stability and scaling gaps. 1/ 📊 Moving past "vibe-based" testing. We’ve built a comprehensive diagnostic benchmark to systematically evaluate whole-body teleoperation.... No more trial-and-error—get the actionable insights needed for true policy optimization. 2/ 👤 Universal Human-to-Robot Mapping. Teleop often breaks when switching operators. OmniClone mitigates biases from hardware fluctuations and, crucially, diverse human body shapes, ensuring high-stability control regardless of the person in the suit. 3/ 🚀 System Optimizations for Whole-body Manipulation Policy. By optimizing for affordability and reproducibility, OmniClone provides the high-fidelity pipeline necessary to collect data and train humanoid whole-body policies at scale. fully The model checkpoints and deploy code are now fully released—welcome to play with it! 📦 📄 Paper: 🌐 Project: 💻 Code:show more

Siyuan Huang

5,150 subscribers

14,109 просмотров • 6 месяцев назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 4

Фото профиля Sitarama Chekuri

Sitarama Chekuri6 месяцев назад

Do you have support for quest vr headset?

Фото профиля Shadow Defense

Shadow Defense6 месяцев назад

so how much is this bot and the teleportation system?

Фото профиля Sanjil Jain 🤖/acc

Sanjil Jain 🤖/acc6 месяцев назад

Scaling data collection is where most humanoid efforts get stuck. The pipeline matters more than the model.

Фото профиля Josh Michael

Josh Michael6 месяцев назад

I want Robots on the street!

Похожие видео

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

The Humanoid Hub

19,563 просмотров • 2 месяцев назад

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,624 просмотров • 10 месяцев назад

So cool...a humanoid just drove a go-kart for real. A Unitree G1 lowers itself into the tight cockpit, hands on the wheel, feet on the pedals, then runs the track， even drifts. It shows constrained whole-body loco-manipulation, mobile pick-and-place, and hand-eye-foot coordination in one sequence. This comes from China’s Symbiosis Robotics and their Direct Perception Control (DPC) model. It skips the usual high-level goals + tracker stack, mapping vision, language, body state and feedback straight to joint and hand targets. The focus is whole-body coordination for loco-manipulation. Interestingly, data diversity...Training used 15,010 hours of joint-aligned data-- 6,781h human egocentric, 4,024h armed robots, 3,660h wheeled humanoids, 545h bipedal. Teleop, head-mounted and egocentric sources were all converted into time-aligned joint trajectories across embodiments. A clean test that packs perception, balance and continuous whole-body control into one task chain ,and leaves room for other real-world scenarios.

So cool...a humanoid just drove a go-kart for real. A Unitree G1 lowers itself into the tight cockpit, hands on the wheel, feet on the pedals, then runs the track， even drifts. It shows constrained whole-body loco-manipulation, mobile pick-and-place, and hand-eye-foot coordination in one sequence. This comes from China’s Symbiosis Robotics and their Direct Perception Control (DPC) model. It skips the usual high-level goals + tracker stack, mapping vision, language, body state and feedback straight to joint and hand targets. The focus is whole-body coordination for loco-manipulation. Interestingly, data diversity...Training used 15,010 hours of joint-aligned data-- 6,781h human egocentric, 4,024h armed robots, 3,660h wheeled humanoids, 545h bipedal. Teleop, head-mounted and egocentric sources were all converted into time-aligned joint trajectories across embodiments. A clean test that packs perception, balance and continuous whole-body control into one task chain ,and leaves room for other real-world scenarios.

CyberRobo

76,039 просмотров • 28 дней назад

I have been cleaning my daughter's mess for more than two years now. Last weekend our robot came to home to do the job for me. 🤖 Our new work on visual whole-body control learns a policy to coordinate the robot legs and arms for mobile manipulation. See how the legs bend on grasping objects on the ground based on visual inputs autonomously. (It is NOT teleoperation!) We again adopt a Sim2Real approach and our policy generalizes to beaches, forests, and streets in San Diego. 👇🧵

I have been cleaning my daughter's mess for more than two years now. Last weekend our robot came to home to do the job for me. 🤖 Our new work on visual whole-body control learns a policy to coordinate the robot legs and arms for mobile manipulation. See how the legs bend on grasping objects on the ground based on visual inputs autonomously. (It is NOT teleoperation!) We again adopt a Sim2Real approach and our policy generalizes to beaches, forests, and streets in San Diego. 👇🧵

Xiaolong Wang

159,462 просмотров • 2 лет назад

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,864 просмотров • 7 месяцев назад

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

AI at Meta

129,278 просмотров • 1 год назад

🤖 Ever dreamed of controlling a humanoid robot to perform complex, long-horizon tasks — using just a single Vision Pro? 🎉 Meet CLONE: a holistic, closed-loop, whole-body teleoperation system for long-horizon humanoid control! 🏃‍♂️🧍 CLONE enables rich and coordinated interactive tasks: 🥊 boxing 🏓 table tennis 🤲 object pickup 📦 room arrangement 🤝 handover … and more! 🌀 Our closed-loop error correction powered by LiDAR odometry ensures precision, while motion-captured demonstrations supercharge policy learning — unlocking the full potential of the G1 robot. 🎥 It’s hard to squeeze the magic into 1 minute — check out the full video demo and project page here: 🔗

🤖 Ever dreamed of controlling a humanoid robot to perform complex, long-horizon tasks — using just a single Vision Pro? 🎉 Meet CLONE: a holistic, closed-loop, whole-body teleoperation system for long-horizon humanoid control! 🏃‍♂️🧍 CLONE enables rich and coordinated interactive tasks: 🥊 boxing 🏓 table tennis 🤲 object pickup 📦 room arrangement 🤝 handover … and more! 🌀 Our closed-loop error correction powered by LiDAR odometry ensures precision, while motion-captured demonstrations supercharge policy learning — unlocking the full potential of the G1 robot. 🎥 It’s hard to squeeze the magic into 1 minute — check out the full video demo and project page here: 🔗

Siyuan Huang

66,986 просмотров • 1 год назад

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

1/🧠Humans are the best robot data source — but video alone misses one thing: force. 2/🙁Tactile gloves capture force — but they're costly and block the real touch manipulation depends on. 3/💪Maybe the future of touch lives on your wrist: surface EMG reads the muscles that cause force — tactile sensing without ever touching a tactile sensor. 4/🔥Want a fully open-source framework — hardware + software — to train your own force-aware learn-from-human-data robot policy? 🚀We introduce ForceBand: Learning Forceful Manipulation with sEMG -- bring force into human videos with sEMG, for force-aware manipulation ⬇️ ✦ Zero-Shot Human-to-Robot Transfer ✦ Force Beyond Vision ✦ Free-Hand Force Sensing ✦ Collect by Anyone, Anytime, Anywhere ✦ Deploy on Any Robot, Any Camera, Any Environment ✦ Open-Source & Low-Cost & Easy-to-Implement Let's squeeze every bit of signal out of human data, and let robots feel the force! 🌐 Website: 📄 Paper: 💻 Code: 🎥 Video: 🧵 1/n

Zhi (Leo) Wang

52,211 просмотров • 2 месяцев назад

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi

Zhengyi “Zen” Luo

47,342 просмотров • 2 лет назад

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 P.S. I put the project page in the comments!

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 P.S. I put the project page in the comments!

Lukas Ziegler

233,153 просмотров • 11 месяцев назад

Introducing Open-TeleVision: with Fully Autonomous policy video👇. We can conduct a long-horizon task with inserting 12 cans nonstop without any interruptions. We offer: 🤖 Highly precise and smooth bimanual manipulation. 📺 Active egocentric vision (with a moving neck) feedback. It is achieved by imitation learning from teleoperation: We propose a VR-based REAL-TIME teleoperation that streams the stereo video observation from the robot camera to the VR device. The robot neck moves as the human head moves, the robot hands move as the human hands move, offering the operator an intuitive experience as the human herself becomes the robot. The devils are all in the details, and how to implement things right: ✅How to perform IK/retargeting for smooth and precise control. ✅How to do all these and also stream stereo video without no latency, all in real time. We released our code here: Active head hardware design: 1/n

Introducing Open-TeleVision: with Fully Autonomous policy video👇. We can conduct a long-horizon task with inserting 12 cans nonstop without any interruptions. We offer: 🤖 Highly precise and smooth bimanual manipulation. 📺 Active egocentric vision (with a moving neck) feedback. It is achieved by imitation learning from teleoperation: We propose a VR-based REAL-TIME teleoperation that streams the stereo video observation from the robot camera to the VR device. The robot neck moves as the human head moves, the robot hands move as the human hands move, offering the operator an intuitive experience as the human herself becomes the robot. The devils are all in the details, and how to implement things right: ✅How to perform IK/retargeting for smooth and precise control. ✅How to do all these and also stream stereo video without no latency, all in real time. We released our code here: Active head hardware design: 1/n

Xiaolong Wang

25,572 просмотров • 2 лет назад

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

467,141 просмотров • 1 год назад

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the motion of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,565 просмотров • 2 лет назад

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 🔗 Find the link here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Build a humanoid robot yourself! 🪚 OpenArm is an open-source humanoid robot. It comes with full CAD, control code, firmware, and simulation tools, everything needed to build, modify, and operate it. The arms are designed to be compliant and backdrivable. Teleoperation is supported, with force feedback and real-time gravity compensation so operators can guide the arm naturally. Super important, on the simulation side, OpenArm works with platforms like MuJoCo and Isaac Sim, letting developers test policies in virtual environments before running on hardware. Assemble it yourself from a kit or get it prebuilt, the goal is accessibility for research labs, small teams, and enthusiasts. The project is run by Enactic in Tokyo, Japan, and aims to lower the barrier for experimenting with dexterous manipulation. Let's put robotics in mainstream! 🔥🔥 🔗 Find the link here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

64,901 просмотров • 4 дней назад

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!

RoboPapers

24,802 просмотров • 5 месяцев назад

Do you even lift? A teleoperation framework that enables full-size humanoids to handle heavy payloads: Like this 175cm, 65kg L7 robot that handles up to 24kg using raw, noisy consumer VR inputs for human intent tracking. Lab demos show the L7 robot walking stably while lifting two 12kg kettlebells or a 10kg water bottle, alongside household tasks like carrying grocery baskets, interacting with a fridge, and serving items, highlighting real-world versatility. The work also validates HEFT on the Unitree G1 platform for noisy VR teleop and high-dynamic whole-body tracking, with open training code, checkpoints, and a project site at Thank you so much for sharing, Qingzhou Lu! 📌 Website & more demos: G1 & L7 training code/checkpoints: ——- Weekly robotics and AI insights. Subscribe free:

Do you even lift? A teleoperation framework that enables full-size humanoids to handle heavy payloads: Like this 175cm, 65kg L7 robot that handles up to 24kg using raw, noisy consumer VR inputs for human intent tracking. Lab demos show the L7 robot walking stably while lifting two 12kg kettlebells or a 10kg water bottle, alongside household tasks like carrying grocery baskets, interacting with a fridge, and serving items, highlighting real-world versatility. The work also validates HEFT on the Unitree G1 platform for noisy VR teleop and high-dynamic whole-body tracking, with open training code, checkpoints, and a project site at Thank you so much for sharing, Qingzhou Lu! 📌 Website & more demos: G1 & L7 training code/checkpoints: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

11,653 просмотров • 2 месяцев назад

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

Sawyer Merritt

624,876 просмотров • 7 месяцев назад

Italy's IIT really has so many humanoid gems. Came across their 2018 Centauro--a centaur-like disaster response robot that combines a quadruped lower body with a humanoid torso and arms. Built for harsh settings like rubble fields or disaster zones, it needs both stable movement over uneven ground and the strength to use human tools for forceful tasks such as breaching doors or clearing obstacles. The four legs deliver terrain-following locomotion and stair climbing, while the upper body handles dexterous, high-force manipulation. The robot blends full-body teleoperation--mapping an operator’s movements directly to the machine, with local autonomous modules for motion and impedance control, supporting supervised or higher-level autonomy when useful. At 1.5m tall and 93kg, it carries roughly 11kg payload per arm with impact-resilient actuators. Battery life is about 2.5 hours. Its hybrid wheeled-legged legs (6 DoF each, with wheels at the ankles acting like rolling hooves) provide better stability than pure bipeds while keeping tool use intuitive. This focus on matching robot morphology to real task demands, rather than defaulting to a standard humanoid form, remains a useful reference. Curious what others think:could hybrid centaur-style platforms play a larger role in future rescue or specialized operations?

Italy's IIT really has so many humanoid gems. Came across their 2018 Centauro--a centaur-like disaster response robot that combines a quadruped lower body with a humanoid torso and arms. Built for harsh settings like rubble fields or disaster zones, it needs both stable movement over uneven ground and the strength to use human tools for forceful tasks such as breaching doors or clearing obstacles. The four legs deliver terrain-following locomotion and stair climbing, while the upper body handles dexterous, high-force manipulation. The robot blends full-body teleoperation--mapping an operator’s movements directly to the machine, with local autonomous modules for motion and impedance control, supporting supervised or higher-level autonomy when useful. At 1.5m tall and 93kg, it carries roughly 11kg payload per arm with impact-resilient actuators. Battery life is about 2.5 hours. Its hybrid wheeled-legged legs (6 DoF each, with wheels at the ankles acting like rolling hooves) provide better stability than pure bipeds while keeping tool use intuitive. This focus on matching robot morphology to real task demands, rather than defaulting to a standard humanoid form, remains a useful reference. Curious what others think:could hybrid centaur-style platforms play a larger role in future rescue or specialized operations?

CyberRobo

41,894 просмотров • 2 месяцев назад

From the Ground to the Skies High-risk operations are now entrusted to humanoid robots, sparing humans the need to risk their lives working at heights of several hundred meters. For example, Robot++’s specialized semi-humanoid platform designed for high-altitude work. Indeed, it is not a fullyhumanoid robot. Its upper body is humanoid in form, allowing for the customized installation of various end effectors; its dual arms work in tandem,welding with one while grinding with the other. Its lower body consists of a wheeled chassis equipped with magnetic adhesion capabilities, enabling movement both vertically and horizontally. TBH,even when operated remotely--from the safety of the ground,former welders are transformed into professional operators, no longer required to place themselves in harm's way.

From the Ground to the Skies High-risk operations are now entrusted to humanoid robots, sparing humans the need to risk their lives working at heights of several hundred meters. For example, Robot++’s specialized semi-humanoid platform designed for high-altitude work. Indeed, it is not a fullyhumanoid robot. Its upper body is humanoid in form, allowing for the customized installation of various end effectors; its dual arms work in tandem,welding with one while grinding with the other. Its lower body consists of a wheeled chassis equipped with magnetic adhesion capabilities, enabling movement both vertically and horizontally. TBH,even when operated remotely--from the safety of the ground,former welders are transformed into professional operators, no longer required to place themselves in harm's way.

CyberRobo

20,952 просмотров • 3 месяцев назад