Загрузка видео...

Не удалось загрузить видео

На главную

Pen spinning on WUJI Hand 2, from simulation training to real-world execution. We use motion-reference tracking and reinforcement learning to bring a simulation-trained policy to the physical hand. Code and setup guides are open source for developers and researchers to build on. Code:

wuji_global's profile picture

WUJI

3,775 subscribers

12,065 просмотров • 6 дней назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 3

Фото профиля PRUTHVI GEEDH
PRUTHVI GEEDH6 дней назад

Always fascinated with @wuji_global

Фото профиля Kuldeep Pisda
Kuldeep Pisda6 дней назад

Shipping the pen itself as STEP and 3MF files with the tag assets is what makes this reproducible. Most sim-to-real repos assume you own the same object. How sensitive is the spin to pen mass?

Фото профиля zhang hao
zhang hao6 дней назад

Currently the Most advanced humanoid robot hand

Похожие видео

MINT is now open source. From first-person RGB video, MINT reconstructs 3D camera trajectories and bimanual hand motion in a shared world coordinate system. Reconstructed human hand motion can also be retargeted to WUJI Hand. The release includes model weights, training and inference code, visualization and evaluation tools, EgoPipeline reference code, and structured annotations for 560,649 episodes covering 1,021 hours of first-person video. Project: Code: #MINT #EmbodiedAI #Robotics #WUJIHand
1:21

MINT is now open source. From first-person RGB video, MINT reconstructs 3D camera trajectories and bimanual hand motion in a shared world coordinate system. Reconstructed human hand motion can also be retargeted to WUJI Hand. The release includes model weights, training and inference code, visualization and evaluation tools, EgoPipeline reference code, and structured annotations for 560,649 episodes covering 1,021 hours of first-person video. Project: Code: #MINT #EmbodiedAI #Robotics #WUJIHand

WUJI

23,181 просмотров • 25 дней назад

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 просмотров • 1 год назад

The next frontier of AI is physical. 🤖 From simulation to synthetic data, developers are training robots to understand the real world with NVIDIA Cosmos, Isaac simulation frameworks and open-source physics engine, Newton. Watch the video to see how partners like Hon Hai Technology Group (Foxconn), Hexagon, Humanoid, Noble Machines, and Skild AI are building the future. 🎥
0:51

The next frontier of AI is physical. 🤖 From simulation to synthetic data, developers are training robots to understand the real world with NVIDIA Cosmos, Isaac simulation frameworks and open-source physics engine, Newton. Watch the video to see how partners like Hon Hai Technology Group (Foxconn), Hexagon, Humanoid, Noble Machines, and Skild AI are building the future. 🎥

NVIDIA Robotics

21,664 просмотров • 6 месяцев назад

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.
1:01

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

Praveen Kumar Verma

138,017 просмотров • 1 месяц назад

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:
0:24

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

Jim Fan

909,423 просмотров • 2 лет назад

Your next robotics project could start with these steps. Today, we're open-sourcing Asimov 1's locomotion policy and the training code behind it, giving developers a foundation to build on. Alongside the trained policy checkpoint, we're sharing our Isaac Lab-based training code, including reward definitions, hardware-tuned actuator configurations, and domain randomization settings. That foundation gives you a starting point for your own experiments. Adapt the training setup to hardware changes. Explore different walking styles. Build toward new capabilities. We've taken the first steps. Build what comes next. Asimov 1 locomotion:
2:39

Your next robotics project could start with these steps. Today, we're open-sourcing Asimov 1's locomotion policy and the training code behind it, giving developers a foundation to build on. Alongside the trained policy checkpoint, we're sharing our Isaac Lab-based training code, including reward definitions, hardware-tuned actuator configurations, and domain randomization settings. That foundation gives you a starting point for your own experiments. Adapt the training setup to hardware changes. Explore different walking styles. Build toward new capabilities. We've taken the first steps. Build what comes next. Asimov 1 locomotion:

Asimov

57,341 просмотров • 10 дней назад

Humanoid robot development is too fragmented to scale. 🤖 With NVIDIA Isaac GR00T, developers unify simulation, data, and training in one open platform, cutting setup time from days to hours. See how Isaac GR00T helps research teams move faster from robot bring-up to skill development and real-world validation. Watch now 📺
0:38

Humanoid robot development is too fragmented to scale. 🤖 With NVIDIA Isaac GR00T, developers unify simulation, data, and training in one open platform, cutting setup time from days to hours. See how Isaac GR00T helps research teams move faster from robot bring-up to skill development and real-world validation. Watch now 📺

NVIDIA Robotics

22,669 просмотров • 3 месяцев назад

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.
0:34

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

Carlo Sferrazza

218,382 просмотров • 10 месяцев назад

🤖🤖🤖 Following RoboVerse, we introduce another work focused on Robotic Tactile Simulation - Taccel Simulator. Taccel is a high-performance simulation platform for vision-based tactile sensors and robots. 🚀🚀🚀 Boosted by Nvidia Warp, we optimize Taccel with highly parallelized simulations and support 900fps simulation with 4k+ parallel training envs. 🤝🤝🤝 Taccel is designed with user-friendly APIs and is easy to use. We open-sourced all the code and documentation. Feel free to try! Project: Preprint: Code:
2:27

🤖🤖🤖 Following RoboVerse, we introduce another work focused on Robotic Tactile Simulation - Taccel Simulator. Taccel is a high-performance simulation platform for vision-based tactile sensors and robots. 🚀🚀🚀 Boosted by Nvidia Warp, we optimize Taccel with highly parallelized simulations and support 900fps simulation with 4k+ parallel training envs. 🤝🤝🤝 Taccel is designed with user-friendly APIs and is easy to use. We open-sourced all the code and documentation. Feel free to try! Project: Preprint: Code:

Siyuan Huang

10,706 просмотров • 1 год назад

Excited to announce RoboCasa, a large-scale simulation framework of everyday tasks! We use generative AI tools to create diverse objects, scenes, and tasks. Simulation plays a pivotal role in our Data Pyramid for training generalist robots. Open-source at
0:34

Excited to announce RoboCasa, a large-scale simulation framework of everyday tasks! We use generative AI tools to create diverse objects, scenes, and tasks. Simulation plays a pivotal role in our Data Pyramid for training generalist robots. Open-source at

Yuke Zhu

141,540 просмотров • 2 лет назад

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.
1:26

Jim Fan says NVIDIA trained humanoid robots to walk and move like humans -- zero-shot transfer from simulation to the real world. 10 years of learning, compressed into 2 hours of simulation. Turns out you don’t need a giant model to master motion. “1.5 million parameters, not billion, to capture the subconscious processing of the human body.” Big embodiment, small brain.

vitrupo

714,883 просмотров • 1 год назад

Given the strong community adoption and real-world deployment of MolmoAct2 on YAM, we're introducing zero-shot evaluation of MolmoAct2 Bimanual YAM in simulation. Now you can test out our models without a real-world robot and build on them! Code: Simulation built on Maniskill!
0:20

Given the strong community adoption and real-world deployment of MolmoAct2 on YAM, we're introducing zero-shot evaluation of MolmoAct2 Bimanual YAM in simulation. Now you can test out our models without a real-world robot and build on them! Code: Simulation built on Maniskill!

Jiafei Duan

37,276 просмотров • 3 месяцев назад

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.
1:22

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,585 просмотров • 4 месяцев назад

It feels like an autonomous living entity ETH Zurich researchers taught the Wuji Hand to move independently by treating its fingers as legs. A reinforcement learning policy learns how to coordinate the fingers for locomotion, while onboard sensing and control let the hand adapt its movements in the real world. The result is the Wuji hand that can crawl across gravel paths, lawns and stone paving, or even play golf and games. This opens up the possibility of robots moving and operating in narrow spaces under desks, at the base of cabinets, inside keyboard trays, equipment gaps, or even under beds. Imagine the future--detached from its humanoid body, a robotic hand could still move independently and execute tasks based on commands.
1:29

It feels like an autonomous living entity ETH Zurich researchers taught the Wuji Hand to move independently by treating its fingers as legs. A reinforcement learning policy learns how to coordinate the fingers for locomotion, while onboard sensing and control let the hand adapt its movements in the real world. The result is the Wuji hand that can crawl across gravel paths, lawns and stone paving, or even play golf and games. This opens up the possibility of robots moving and operating in narrow spaces under desks, at the base of cabinets, inside keyboard trays, equipment gaps, or even under beds. Imagine the future--detached from its humanoid body, a robotic hand could still move independently and execute tasks based on commands.

CyberRobo

18,947 просмотров • 18 дней назад

We Luis Felipe Casas have successfully deployed an in-hand manipulation policy to the LEAP hand following RL training in MuJoCo Playground. Sim-to-real isn’t as straightforward as it seems. Policy design and real-world tuning matter. Working on improving the policy further.
0:49

We Luis Felipe Casas have successfully deployed an in-hand manipulation policy to the LEAP hand following RL training in MuJoCo Playground. Sim-to-real isn’t as straightforward as it seems. Policy design and real-world tuning matter. Working on improving the policy further.

Yu Xiang

11,130 просмотров • 6 месяцев назад

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla
1:06:07

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 просмотров • 8 месяцев назад

I placed 🥈 2nd in the LeHome Challenge (at IEEE ICRA 2026) earlier this month, and before that I was 🥇 1st of 62 teams in the simulation round. Now I am sharing my solution — with a detailed logic walkthrough and open-source code. The task was to teach a cheap two-armed robot to fold different garments in simulation and on a real robot. I trained a VLA policy with an RL loop to make it work. Let's break it down 👇
0:33

I placed 🥈 2nd in the LeHome Challenge (at IEEE ICRA 2026) earlier this month, and before that I was 🥇 1st of 62 teams in the simulation round. Now I am sharing my solution — with a detailed logic walkthrough and open-source code. The task was to teach a cheap two-armed robot to fold different garments in simulation and on a real robot. I trained a VLA policy with an RL loop to make it work. Let's break it down 👇

Ilia

29,522 просмотров • 3 месяцев назад

Evolving dexterity with GPT-6 Astra 🖐️ Been trying Astra recently. Its zero-shot dexterous manipulation is already quite surprising. More interesting is seeing it learn and improve through simulation training, from pen spinning and Rubik's Cube to hammer use. The real goal would be to evolve this dexterity in the real world.
0:34

Evolving dexterity with GPT-6 Astra 🖐️ Been trying Astra recently. Its zero-shot dexterous manipulation is already quite surprising. More interesting is seeing it learn and improve through simulation training, from pen spinning and Rubik's Cube to hammer use. The real goal would be to evolve this dexterity in the real world.

Jianglong Ye

109,195 просмотров • 17 дней назад