Loading video...

Video Failed to Load

Go Home

Want to experience humanoid behavior foundation model in your browser? SONIC Web demo now live at Featuring: - Motion tracking - Text to motion using Kimodo Try it out!

zhengyiluo's profile picture

Zhengyi “Zen” Luo

6,064 subscribers

38,564 views • 3 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

figured out how to do motion blurs in the browser using svg filters and css
0:35

figured out how to do motion blurs in the browser using svg filters and css

ravarcheon

24,707 views • 8 months ago

robust weight transfer in the web browser. a little demo i made to try. check it out! link below 👇
0:31

robust weight transfer in the web browser. a little demo i made to try. check it out! link below 👇

Sent From Space 👽

12,631 views • 1 year ago

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.
2:39

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.

The Humanoid Hub

31,445 views • 8 months ago

It’s here. Veo 3 is now live for all Pro users in LTX Studio. Text-to-video just leveled up. More motion, more detail, more cinematic control. Try it today.
0:41

It’s here. Veo 3 is now live for all Pro users in LTX Studio. Text-to-video just leveled up. More motion, more detail, more cinematic control. Try it today.

LTX.io

667,773 views • 1 year ago

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!
59:44

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!

RoboPapers

24,802 views • 3 months ago

The humanoid robot Lingxi X2 can easily dodge thrown balls… powered by AGIBOT’s newly launched AGILE perception-motion foundation model. With this motion intelligence, it can avoid interference and make real-time perceptual adjustments when working in complex environments. I’m curious whether it could dodge a human’s flying kick? (They always seem to kick humanoid robots during testing.)
0:24

The humanoid robot Lingxi X2 can easily dodge thrown balls… powered by AGIBOT’s newly launched AGILE perception-motion foundation model. With this motion intelligence, it can avoid interference and make real-time perceptual adjustments when working in complex environments. I’m curious whether it could dodge a human’s flying kick? (They always seem to kick humanoid robots during testing.)

CyberRobo

32,796 views • 2 months ago

A professional motion studio now runs entirely in your browser. Meet Lottie Creator 2.0: describe an animation & Motion Copilot builds it, shape every curve in the new graph editor, add interactivity with State Machines & export production-ready Lottie. No installs, nothing to maintain. Try it free 👇
1:19

A professional motion studio now runs entirely in your browser. Meet Lottie Creator 2.0: describe an animation & Motion Copilot builds it, shape every curve in the new graph editor, add interactivity with State Machines & export production-ready Lottie. No installs, nothing to maintain. Try it free 👇

LottieFiles

27,909 views • 15 days ago

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js.🤗 Try it out yourself! Demo + source code below 👇
0:32

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js.🤗 Try it out yourself! Demo + source code below 👇

Xenova

19,666 views • 1 year ago

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.
1:11

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

The Humanoid Hub

25,125 views • 8 months ago

Web Search on ASI-1 Mini now live🚨 Web search combined with agents is a game changer, live search with source identification. Watch the demo to see it in action 👇 Try it out:
0:28

Web Search on ASI-1 Mini now live🚨 Web search combined with agents is a game changer, live search with source identification. Watch the demo to see it in action 👇 Try it out:

Fetch.ai

23,569 views • 1 year ago

Want to achieve extreme performance in motion tracking—and go beyond it? Our preprint tech report is now online, with open-source code available!
0:49

Want to achieve extreme performance in motion tracking—and go beyond it? Our preprint tech report is now online, with open-source code available!

Qiayuan Liao

365,767 views • 11 months ago

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
1:32

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,208 views • 8 months ago

End to end, autonomous whole body motion generation. You can see this demo in “live”(not video) at ICRA2024.
0:27

End to end, autonomous whole body motion generation. You can see this demo in “live”(not video) at ICRA2024.

Tetsuya Ogata / 尾形哲也

42,928 views • 2 years ago

Want free, local AI in your browser? Chrome's built-in Prompt API is ready to try and is also shipped for Chrome Extensions. Powered by Gemini Nano. Demo featuring Y Combinator 😅
3:20

Want free, local AI in your browser? Chrome's built-in Prompt API is ready to try and is also shipped for Chrome Extensions. Powered by Gemini Nano. Demo featuring Y Combinator 😅

Addy Osmani

45,618 views • 11 months ago

Higgsfield just added Flux Kontext Now anyone can edit any pic, drop your product, add motion + Speak. All in single browser tab.🤯 8 wild demos & how to try:
0:29

Higgsfield just added Flux Kontext Now anyone can edit any pic, drop your product, add motion + Speak. All in single browser tab.🤯 8 wild demos & how to try:

Min Choi

299,528 views • 1 year ago

Yong Su ported Evan Wallace's classic WebGL water demo to WebGPU. Try it right in your browser!
0:20

Yong Su ported Evan Wallace's classic WebGL water demo to WebGPU. Try it right in your browser!

80 LEVEL

428,389 views • 6 months ago

Yong Su ported Evan Wallace's classic WebGL water demo to WebGPU. Try it right in your browser:
0:39

Yong Su ported Evan Wallace's classic WebGL water demo to WebGPU. Try it right in your browser:

80 LEVEL

35,418 views • 6 months ago

Grok Imagine Video 1.5 is live on AITOPIA. @xAI's latest video model is here. Generate from text or image. Native audio with dialogue, music, and effects. Realistic motion and strong prompt adherence. Try it now →
0:29

Grok Imagine Video 1.5 is live on AITOPIA. @xAI's latest video model is here. Generate from text or image. Native audio with dialogue, music, and effects. Realistic motion and strong prompt adherence. Try it now →

AITOPIA

46,050 views • 1 month ago