Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Ever wonder how we got our humanoids to walk naturally using a motion tracker? Here is the secret: We have a world-class, game-engine-ready, real-time motion generator running under the hood! Graphics 🤝 Robotics Check out the full thread here 👇

zhengyiluo's profile picture

Zhengyi “Zen” Luo

6,058 subscribers

15,229 Aufrufe • vor 4 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
1:32

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,399 Aufrufe • vor 9 Monaten

Ever wonder just how we got here? To the sleepers we have now?
5:36

Ever wonder just how we got here? To the sleepers we have now?

Trucker Dan

13,922 Aufrufe • vor 8 Monaten

We see a lot of slow motion Michael Jordan, here is what young Mike was doing to the league in real time speed 🔥 (fanofthegame TT)
0:54

We see a lot of slow motion Michael Jordan, here is what young Mike was doing to the league in real time speed 🔥 (fanofthegame TT)

Jacob

71,476 Aufrufe • vor 3 Monaten

here's how we did it and the open sourced HyperFrames code in thread 🧵 i did a quick screen recording as a quick explainer our open source framework & motion graphics skills:
0:36

here's how we did it and the open sourced HyperFrames code in thread 🧵 i did a quick screen recording as a quick explainer our open source framework & motion graphics skills:

Bin Liu

32,579 Aufrufe • vor 1 Monat

This is the best champions league game we played under Xavi. Crazy how we didn’t walk out with a win.
4:50

This is the best champions league game we played under Xavi. Crazy how we didn’t walk out with a win.

ELI LEE

514,341 Aufrufe • vor 2 Monaten

We see so many slow motion clips of Jordan, here is all footage in real time speed. We forgot Michael was a BLUR on the court 👀🔥 (Queztheplug IG)
1:01

We see so many slow motion clips of Jordan, here is all footage in real time speed. We forgot Michael was a BLUR on the court 👀🔥 (Queztheplug IG)

Jacob

64,388 Aufrufe • vor 26 Tagen

Here is the Full Video👇👇 We have to fight for our Country #citizenoutreach
1:18

Here is the Full Video👇👇 We have to fight for our Country #citizenoutreach

UMUKUNDWA Lilly

15,174 Aufrufe • vor 1 Jahr

In response to our motion to unimpound all of the discovery we have, Cosgrove introduced into evidence an article on Ratchet Madness. Here is the judges reaction.
0:27

In response to our motion to unimpound all of the discovery we have, Cosgrove introduced into evidence an article on Ratchet Madness. Here is the judges reaction.

Aidan Kearney

85,505 Aufrufe • vor 1 Jahr

Quantum threat: how long do we have? Quantus's Yuvi Lightman | Quantus highlights differing perspectives on how much time we have before the world's cryptography needs to be upgraded. Check out the full interview here:
0:31

Quantum threat: how long do we have? Quantus's Yuvi Lightman | Quantus highlights differing perspectives on how much time we have before the world's cryptography needs to be upgraded. Check out the full interview here:

Moby from Whale Coin Talk

21,437 Aufrufe • vor 3 Monaten

We have an amazing surprise! 😊♥️ We got to film with the wonderful Hearts2Hearts and we played a super fun game of Know or No! Check out the video here: #Hearts2Hearts #하츠투하츠 #H2H #STYLE #Hearts2Hearts_STYLE
0:44

We have an amazing surprise! 😊♥️ We got to film with the wonderful Hearts2Hearts and we played a super fun game of Know or No! Check out the video here: #Hearts2Hearts #하츠투하츠 #H2H #STYLE #Hearts2Hearts_STYLE

BuzzFeed UK

63,188 Aufrufe • vor 1 Jahr

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!
0:53

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!

Zipeng Fu

230,266 Aufrufe • vor 2 Jahren

The time to release the 1871 thread is here. Who’s ready to find out we aren’t (weren’t) free. And I’ve got alllllllllll the corresponding government and congressional and treaty citations/sauce you can handle
2:46

The time to release the 1871 thread is here. Who’s ready to find out we aren’t (weren’t) free. And I’ve got alllllllllll the corresponding government and congressional and treaty citations/sauce you can handle

IET 17

310,918 Aufrufe • vor 2 Jahren

Hey Cardano! Want to see Ouroboros-Leios working? We have a demo below. This is real, running code — not theory. Watch the full demo here 👇 #Cardano #Leios #Ouroboros #Treasury
7:16

Hey Cardano! Want to see Ouroboros-Leios working? We have a demo below. This is real, running code — not theory. Watch the full demo here 👇 #Cardano #Leios #Ouroboros #Treasury

Carlos Lopez de Lara

62,804 Aufrufe • vor 3 Monaten

Game results reflect practice habits. "We have got to understand - we win the games today. We win the games before we play them. Then you deserve to win. When we come in here, it's a plan to get ready to win the game."
0:25

Game results reflect practice habits. "We have got to understand - we win the games today. We win the games before we play them. Then you deserve to win. When we come in here, it's a plan to get ready to win the game."

The Winning Difference

42,130 Aufrufe • vor 1 Jahr

The Train-to-Earn era in robotics is here. Train robots. Earn rewards. All in under a minute. We challenged Axis Robotics and Inverted Lambda on the Axis Robotics training simulator. One of them wasn’t expecting the result… 👇
1:21

The Train-to-Earn era in robotics is here. Train robots. Earn rewards. All in under a minute. We challenged Axis Robotics and Inverted Lambda on the Axis Robotics training simulator. One of them wasn’t expecting the result… 👇

Base APAC

13,499 Aufrufe • vor 1 Monat

It is a miracle that we can understand and create a model of the universe in our minds, and I can't think of a better way to spend my time here. #wonder #science #thepoetryofreality The full episode is here:
0:33

It is a miracle that we can understand and create a model of the universe in our minds, and I can't think of a better way to spend my time here. #wonder #science #thepoetryofreality The full episode is here:

Richard Dawkins

124,660 Aufrufe • vor 2 Jahren

Over a million Sudanese refugees have fled to Chad. Aid is running out. Disease is spreading. And the world isn’t watching. On Talk to Al Jazeera, we ask: Is the humanitarian system collapsing in silence? Watch the full episode here:
0:52

Over a million Sudanese refugees have fled to Chad. Aid is running out. Disease is spreading. And the world isn’t watching. On Talk to Al Jazeera, we ask: Is the humanitarian system collapsing in silence? Watch the full episode here:

Talk to Al Jazeera

80,221 Aufrufe • vor 1 Jahr

We built a real-time multiplayer game generated entirely by a neural network—and now you can actually play it. In collaboration with Modal, we just launched the live demo for MultiGen, our diffusion-based multiplayer game engine. Grab some friends and try it here 👇
0:23

We built a real-time multiplayer game generated entirely by a neural network—and now you can actually play it. In collaboration with Modal, we just launched the live demo for MultiGen, our diffusion-based multiplayer game engine. Grab some friends and try it here 👇

Gordon Wetzstein

26,720 Aufrufe • vor 4 Monaten

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi
1:15

🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform actions like pick and place, walking, kicking, boxing, etc - 💡Unleash the potential of humanoids with human cognitive skills and adaptability 🔗: 📄: 🎬: H2O proposes: - A scalable retargeting framework for obtaining large-scale humanoid motion dataset, intelligently filtering out infeasible motion for the humanoid embodiment. - We train a full-body motion imitator (similar to PHC) and deploy to the real world zero-shot. - Using this framework, we enable real-time teleoperation of a humanoid via a human operator and webcam, performing skills such as pick and place, kicking, walking strollers, etc. Team: Tairan He Zhengyi “Zen” Luo Wenli Xiao @ChongZitaZhang Kris Kitani Changliu Liu Guanya Shi

Zhengyi “Zen” Luo

47,323 Aufrufe • vor 2 Jahren