Загрузка видео...

Не удалось загрузить видео

На главную

Humanoid robots playing table tennis fully autonomously. The 'HITTER' system combines a model-based planner with a reinforcement learning (RL) whole-body controller. It is fully autonomous but relies on an external sensing system. A 9-camera OptiTrack motion capture setup tracks the ball, which is covered with reflective markers, to predict...

TheHumanoidHub's profile picture

The Humanoid Hub

112,549 subscribers

91,896 просмотров • 11 месяцев назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 просмотров • 1 год назад

Researchers developed a system called LATENT that teaches humanoid robots to play tennis using small pieces of human movement data. It learns basic motion fragments, then corrects and combines them to create natural movements and swings. The system was tested on the Unitree G1 humanoid robot, enabling it to hit balls and rally with human players.
0:22

Researchers developed a system called LATENT that teaches humanoid robots to play tennis using small pieces of human movement data. It learns basic motion fragments, then corrects and combines them to create natural movements and swings. The system was tested on the Unitree G1 humanoid robot, enabling it to hit balls and rally with human players.

Space and Technology

45,748 просмотров • 3 месяцев назад

🎾Your humanoid tennis player is here!🤖 Introducing LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data) — the world’s first real-time whole-body planning and control algorithm for athletic humanoid tennis. For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion. This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction. #Galbot #Robotics #Innovation #AI #Technology #Humanoid
1:25

🎾Your humanoid tennis player is here!🤖 Introducing LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data) — the world’s first real-time whole-body planning and control algorithm for athletic humanoid tennis. For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion. This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction. #Galbot #Robotics #Innovation #AI #Technology #Humanoid

Galbot

45,902 просмотров • 5 месяцев назад

🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗:
1:27

🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗:

Tairan He

96,738 просмотров • 2 лет назад

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."
3:25

NEWS: Humanoid robotics company Figure has released Helix 02, what they claim in their most capable humanoid model yet. "A single neural system that controls the full body directly from pixels, enabling dexterous, long horizon autonomy across an entire room: • Autonomous, long‑horizon loco-manipulation: Helix 02 unloads and reloads a dishwasher across a full-sized kitchen - a four-minute, end-to-end autonomous task that integrates walking, manipulation, and balance with no resets and no human intervention. We believe this is the longest horizon, most complex task completed autonomously by a humanoid robot to date. • All sensors in. All actuators out: Helix 02 connects every onboard sensor - vision, touch, and proprioception - directly to every actuator through a single unified visuomotor neural network. • Human-like whole body control from human data: All results are enabled by System 0, a learned whole‑body controller trained on over 1,000 hours of human motion data and sim‑to‑real reinforcement learning. System 0 replaces 109,504 lines of hand‑engineered C++ with a single neural prior for stable, natural motion. • New classes of dexterity: With Figure 03’s embedded tactile sensing and palm cameras, Helix 02 performs manipulation that was previously out of reach: extracting individual pills, dispensing precise syringe volumes, and singulating small, irregular objects from clutter despite self‑occlusion. Helix 02 is trained on over 1,000 hours of human motion data and integrates vision, touch, and proprioception."

Sawyer Merritt

624,770 просмотров • 6 месяцев назад

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.
2:48

X-Humanoid unveiled TG-VLA, which it calls the world's first whole-body VLA framework for full-size humanoid robots. The goal: move humanoids past "mobile dual-arm systems" to human-like, whole-body coordinated operation. Built on three technologies: - HEX: an open-source full-size humanoid VLA that learns across embodiments and coordinates the whole body. Claimed SOTA on real-world whole-body manipulation - HAF-VLA: hierarchical action flow that decomposes whole-body motion into subtasks. Best or joint-best on 6 benchmarks - DSRL-DCT: online RL for high-DoF humanoids in a compressed latent space. 100% success on mobile manipulation HEX is open-source.

The Humanoid Hub

19,563 просмотров • 1 месяц назад

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.
1:10

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

AI at Meta

129,182 просмотров • 1 год назад

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n
1:54

The power of generative models — now embodied in humanoids. Announcing DreamControl –– After a year-long research effort at General Robotics — we present a scalable framework for whole-body humanoid control that fuses diffusion priors with reinforcement learning to unlock real-world scene interaction. Diffusion + RL → natural whole-body skills on real robots. DreamControl enables humanoids to move beyond locomotion demos → performing natural, human-like skills such as –– Picking & lifting objects, Opening drawers & doors, Precise punching, kicking, and jumping, Bimanual manipulation tasks Our key innovation: a diffusion prior over human motion that guides RL, eliminating the need for massive teleoperation datasets, and producing motions that look human while transferring to real hardware. Trained purely in simulation, deployed on the Unitree G1 humanoid, DreamControl policies run in real time, bridging sim-to-real with unprecedented naturalness. We leverage a novel hybrid edge + cloud infrastructure that runs RL-trained policies on the edge backed by powerful AI models running in the cloud This is the next step in General Robotics’ journey toward general-purpose humanoid assistants that interact, adapt, and assist autonomously. Paper: Blog: 1/n

Ashish Kapoor

118,133 просмотров • 11 месяцев назад

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.
0:45

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 просмотров • 10 месяцев назад

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.
0:05

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 просмотров • 6 месяцев назад

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.
0:28

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

gensyn

228,906 просмотров • 1 год назад

A new humanoid robot Spirit AI from China releases a new generation of wheeled humanoid robot Moz1 It is the first embodied intelligent robot with high-precision force control of the whole body, with 26 degrees of freedom, built based on the world's highest power density integrated force control joint. It takes speed, precision and safety to a higher level. It is worth mentioning that Moz1 is driven by SpiritAI's latest OneTwoVLA. It only needs one model to think and operate at the same time, and correct errors autonomously, just like humans, without separating the two systems of "thinking planning" and "coordinated execution". One system is seamlessly connected, which is conducive to dealing with complex, personalized, and immediate tasks, especially in home and care applications.
1:00

A new humanoid robot Spirit AI from China releases a new generation of wheeled humanoid robot Moz1 It is the first embodied intelligent robot with high-precision force control of the whole body, with 26 degrees of freedom, built based on the world's highest power density integrated force control joint. It takes speed, precision and safety to a higher level. It is worth mentioning that Moz1 is driven by SpiritAI's latest OneTwoVLA. It only needs one model to think and operate at the same time, and correct errors autonomously, just like humans, without separating the two systems of "thinking planning" and "coordinated execution". One system is seamlessly connected, which is conducive to dealing with complex, personalized, and immediate tasks, especially in home and care applications.

CyberRobo

26,755 просмотров • 1 год назад

Picking up laundry with a 3D printed robot arm and LeRobot. It operates using SO-ARM101 arm mounted on a mobile base, relying on just a wrist and base camera. The base is controlled with an Xbox controller by a human operator. The arm is fully autonomous thanks to the ACT model. The robot can generalize to previously unseen environments. The technical stack: - Model: Action Chunking Transformer (ACT) with Temporal Ensemble - Data: Learned via teleoperation including negative examples and failure recoveries - Framework: Built and trained locally using LeRobot on a single consumer GPU in 3.5h This is a perfect example that real-world robotics isn't about perfect lab conditions - it's about messy living rooms, confusing shadows, and learning to recover from mistakes. And with open tools, anyone can tackle these edge cases right from their desk.
1:55

Picking up laundry with a 3D printed robot arm and LeRobot. It operates using SO-ARM101 arm mounted on a mobile base, relying on just a wrist and base camera. The base is controlled with an Xbox controller by a human operator. The arm is fully autonomous thanks to the ACT model. The robot can generalize to previously unseen environments. The technical stack: - Model: Action Chunking Transformer (ACT) with Temporal Ensemble - Data: Learned via teleoperation including negative examples and failure recoveries - Framework: Built and trained locally using LeRobot on a single consumer GPU in 3.5h This is a perfect example that real-world robotics isn't about perfect lab conditions - it's about messy living rooms, confusing shadows, and learning to recover from mistakes. And with open tools, anyone can tackle these edge cases right from their desk.

LeRobot

67,896 просмотров • 12 дней назад

Israel-based Mentee Robotics has demonstrated a logistics workflow: two MenteeBot V3 humanoids work autonomously to pick and place totes. A Modular Agent System is preferred because it favors real-world robustness and lower compute needs over the End-to-End VLA model. Its architecture is composed of three components: - LLM Planner: Converts instructions into executable Robotic API Language code for reliable task decomposition and error handling. - Perception Stack: Uses pre-trained models (NeRF/3DGS, distilled vision) for scene understanding and navigation. - Control Policies: Reinforcement Learning (RL) models, trained at scale via Sim2Real, generate motor commands, enabling high-accuracy mobile manipulation. Crucially, the robot learns new tasks from a single demonstration in hours. Object tracking uses 3D geometry (STL/URDF) tracked in the video to define the RL reward function. Training is optimized using 'Automatic Curriculum Learning', which autonomously adjusts task difficulty based on robot performance, eliminating manual engineering. All computation runs onboard.
18:21

Israel-based Mentee Robotics has demonstrated a logistics workflow: two MenteeBot V3 humanoids work autonomously to pick and place totes. A Modular Agent System is preferred because it favors real-world robustness and lower compute needs over the End-to-End VLA model. Its architecture is composed of three components: - LLM Planner: Converts instructions into executable Robotic API Language code for reliable task decomposition and error handling. - Perception Stack: Uses pre-trained models (NeRF/3DGS, distilled vision) for scene understanding and navigation. - Control Policies: Reinforcement Learning (RL) models, trained at scale via Sim2Real, generate motor commands, enabling high-accuracy mobile manipulation. Crucially, the robot learns new tasks from a single demonstration in hours. Object tracking uses 3D geometry (STL/URDF) tracked in the video to define the RL reward function. Training is optimized using 'Automatic Curriculum Learning', which autonomously adjusts task difficulty based on robot performance, eliminating manual engineering. All computation runs onboard.

The Humanoid Hub

15,729 просмотров • 8 месяцев назад

The wait is over. Figure 03 is here, "Designed for Helix, the home, and the world at scale." ⦿ Shown testing at home, performing long-horizon tasks fully autonomously (nothing teleoperated). ⦿ A completely redesigned sensory suite and hand system, which is purpose-built to enable Helix, Figure's proprietary VLA AI. ⦿ 9% lighter than Figure 02. ⦿ Features 2kW wireless charging, an improved audio system for voice reasoning, and battery safety advancements. ⦿ Figure established a new supply chain and an entirely new process for manufacturing humanoid robots. ⦿ A new compact camera architecture delivers twice the frame rate, one-quarter the latency, and a 60% wider field of view per camera. ⦿ An embedded palm camera with a wide field of view and low-latency sensing. ⦿ First-generation in-house developed tactile sensors for fingertips, capable of detecting forces as small as three grams of pressure. ⦿ Includes 10 Gbps mmWave data offload capability. ⦿ Better protection against pinch points. Figure's manufacturing facility, BotQ, has a first-generation manufacturing line that will initially be capable of producing up to 12,000 humanoid robots per year, with the goal of producing a total of 100,000 robots over the next four years.
6:19

The wait is over. Figure 03 is here, "Designed for Helix, the home, and the world at scale." ⦿ Shown testing at home, performing long-horizon tasks fully autonomously (nothing teleoperated). ⦿ A completely redesigned sensory suite and hand system, which is purpose-built to enable Helix, Figure's proprietary VLA AI. ⦿ 9% lighter than Figure 02. ⦿ Features 2kW wireless charging, an improved audio system for voice reasoning, and battery safety advancements. ⦿ Figure established a new supply chain and an entirely new process for manufacturing humanoid robots. ⦿ A new compact camera architecture delivers twice the frame rate, one-quarter the latency, and a 60% wider field of view per camera. ⦿ An embedded palm camera with a wide field of view and low-latency sensing. ⦿ First-generation in-house developed tactile sensors for fingertips, capable of detecting forces as small as three grams of pressure. ⦿ Includes 10 Gbps mmWave data offload capability. ⦿ Better protection against pinch points. Figure's manufacturing facility, BotQ, has a first-generation manufacturing line that will initially be capable of producing up to 12,000 humanoid robots per year, with the goal of producing a total of 100,000 robots over the next four years.

The Humanoid Hub

169,159 просмотров • 10 месяцев назад

Tesla just got granted new patent for robotic Knee join and of course I have to make a video about it 😀 This patent application describes a sophisticated robotic knee joint designed by Tesla to emulate the complex movement of a human leg. The invention utilizes a linear actuator and a specialized linkage system to provide a wide range of motion while maintaining energy efficiency. By integrating processing circuitry and a controller, the system can precisely calculate and execute the necessary torque and displacement for fluid mobility. This mechanical design is specifically tailored for use in humanoid robots, allowing them to perform tasks with a more natural and effective gait. The documentation further includes technical illustrations and simulation data to demonstrate how the joint achieves optimal performance through its unique structural configuration.
6:59

Tesla just got granted new patent for robotic Knee join and of course I have to make a video about it 😀 This patent application describes a sophisticated robotic knee joint designed by Tesla to emulate the complex movement of a human leg. The invention utilizes a linear actuator and a specialized linkage system to provide a wide range of motion while maintaining energy efficiency. By integrating processing circuitry and a controller, the system can precisely calculate and execute the necessary torque and displacement for fluid mobility. This mechanical design is specifically tailored for use in humanoid robots, allowing them to perform tasks with a more natural and effective gait. The documentation further includes technical illustrations and simulation data to demonstrate how the joint achieves optimal performance through its unique structural configuration.

Emily

50,333 просмотров • 4 месяцев назад

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
1:32

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,326 просмотров • 9 месяцев назад

AgiBot has formally unveiled its G2 humanoid robot, a system designed to transition into various industries and liberate humans from repetitive labor. G2 features high-performance joints, precision torque sensors, and an advanced spatial perception system, supporting quick deployment and multi-modal voice interaction. ► Factory Floor Performance: The G2 is engineered to industrial standards. In a safety belt lock production line, robots collaborate with human workers, performing tasks like pressing lock cores. The G2 collects production data to continuously train and iterate models (local server deployment ensures data privacy), steadily improving its operational ability. ► Mobility & Safety: The G2 navigates narrow factory aisles using dual LiDAR and full-panorama vision for environment sensing and collision detection. Its chassis is designed to overcome common obstacles (speed bumps, elevator gaps). It supports 24/7 continuous operation via autonomous return-to-charge and battery swapping. ► Humanoid Design Advantage: The G2's design includes a three-degree-of-freedom flexible waist, allowing it to mimic natural human movements like bending and side-leaning. This dramatically expands its operational workspace and enables seamless integration into existing human-centric production lines without costly modifications. ► Advanced Dexterity & Learning (Lab): The new G02 arm is the world's first cross-moment arm, featuring high-precision joint torque sensors that allow it to precisely sense external forces and adjust stiffness, mimicking human hand compliance. Using Real-Machine Reinforcement Learning (RL), the G2 can learn complex, delicate tasks like memory stick insertion in about one hour with minimal human intervention. ► Logistics & Grasping: In logistics sorting, the G2 uses a 19-degree-of-freedom mechanical dexterous hand (20N maximum fingertip force; 35kg capacity for hard objects) equipped with 3D tactile sensors to ensure it grasps securely without damaging items. Its full-body articulation (waist and legs) aids grasping and posture adjustment. ► Model & Data: G2's intelligence is powered by the Go-One Large Embodied Model (VLA architecture: Vision-Language-Latent Action) and the GE-One World Model (vision-centric predictive modeling), trained using the AgiBot Word true-machine dataset (over 500k downloads). ► Service & Interaction: The G2 is deployed as a guide/receptionist in settings like art museums. It uses its high-DOF head, arms, and waist to point to exhibits, maintains eye contact while navigating difficult spaces (chassis walks forward, body faces backward), handles specialized and random queries, and uses proactive safety features (stops movement, issues warnings) when people get too close.
2:49

AgiBot has formally unveiled its G2 humanoid robot, a system designed to transition into various industries and liberate humans from repetitive labor. G2 features high-performance joints, precision torque sensors, and an advanced spatial perception system, supporting quick deployment and multi-modal voice interaction. ► Factory Floor Performance: The G2 is engineered to industrial standards. In a safety belt lock production line, robots collaborate with human workers, performing tasks like pressing lock cores. The G2 collects production data to continuously train and iterate models (local server deployment ensures data privacy), steadily improving its operational ability. ► Mobility & Safety: The G2 navigates narrow factory aisles using dual LiDAR and full-panorama vision for environment sensing and collision detection. Its chassis is designed to overcome common obstacles (speed bumps, elevator gaps). It supports 24/7 continuous operation via autonomous return-to-charge and battery swapping. ► Humanoid Design Advantage: The G2's design includes a three-degree-of-freedom flexible waist, allowing it to mimic natural human movements like bending and side-leaning. This dramatically expands its operational workspace and enables seamless integration into existing human-centric production lines without costly modifications. ► Advanced Dexterity & Learning (Lab): The new G02 arm is the world's first cross-moment arm, featuring high-precision joint torque sensors that allow it to precisely sense external forces and adjust stiffness, mimicking human hand compliance. Using Real-Machine Reinforcement Learning (RL), the G2 can learn complex, delicate tasks like memory stick insertion in about one hour with minimal human intervention. ► Logistics & Grasping: In logistics sorting, the G2 uses a 19-degree-of-freedom mechanical dexterous hand (20N maximum fingertip force; 35kg capacity for hard objects) equipped with 3D tactile sensors to ensure it grasps securely without damaging items. Its full-body articulation (waist and legs) aids grasping and posture adjustment. ► Model & Data: G2's intelligence is powered by the Go-One Large Embodied Model (VLA architecture: Vision-Language-Latent Action) and the GE-One World Model (vision-centric predictive modeling), trained using the AgiBot Word true-machine dataset (over 500k downloads). ► Service & Interaction: The G2 is deployed as a guide/receptionist in settings like art museums. It uses its high-DOF head, arms, and waist to point to exhibits, maintains eye contact while navigating difficult spaces (chassis walks forward, body faces backward), handles specialized and random queries, and uses proactive safety features (stops movement, issues warnings) when people get too close.

RoboHub🤖

46,733 просмотров • 10 месяцев назад