Загрузка видео...

Не удалось загрузить видео

На главную

Command humanoids *directly* with natural language? Introducing LangWBC, a generative, end-to-end policy that turns natural language into real-world whole-body humanoid control! 💬→🦿Smooth, robust, surprisingly intuitive! See more 👉 #RSS2025

ZhongyuLi4's profile picture

Zhongyu Li

1,497 subscribers

14,142 просмотров • 1 год назад •via X (Twitter)

Наука и технологииОбразование#RSS2025
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 9

Фото профиля Zhongyu Li
Zhongyu Li1 год назад

Intuition of LangWBC: Map language to whole-body motion, using a CVAE for its structured latent space→Enables generalization to linguistic variations & unseen latents! 1. Train lang.-agnostic RL experts for motion tracking 2. Distill into a single e2e policy conditioned on lang.

Фото профиля Zhongyu Li
Zhongyu Li1 год назад

Real-world deployment shows LangWBC in action: ✅ Agile, versatile whole-body motion ✅ Smooth transitions—e.g., walking ➡️ running ✅ Emergent motions via latent space blending ("walk" + "sidestep" = diagonal!) ✅ Easy integration with LLMs for natural interaction

Фото профиля Zhongyu Li
Zhongyu Li1 год назад

📄 This work has been accepted to #RSS2025! Check out the full paper here: Big thanks to our collaborators @Shaoyy2003 @x_h_ucb @qiayuanliao @t_k_233 @KoushilSreenath and everyone who supported this work!

Фото профиля TXYZ
TXYZ1 год назад

⏰ Countdown alert! 🚀 Get free access to our new AI Writing feature before Dec 1st! Don't miss your chance to explore its power. Try it now—no cost, no catch! 🔗

Фото профиля LECCA Intern (Ø,G) 🍜
LECCA Intern (Ø,G) 🍜1 год назад

Language-to-motion in one shot. LangWBC feels like GPT for humanoids.

Фото профиля Sara Orbgirl
Sara Orbgirl1 год назад

Imagine commanding robots with words, like a sci-fi dream come true! 🌠 Who knew talking could create motion? Impressive work! 🤖

Фото профиля Shi Soul
Shi Soul1 год назад

seems the feasible motions are too few( walk, run, wave) can we do sth more?

Фото профиля Weishuai Zeng
Weishuai Zeng1 год назад

Nice work Zhongyu! Congratulations~ May I know how do you obtain base linear velocity in real world as you mention in your paper? QwQ

Фото профиля Sanjar
Sanjar1 год назад

Great work! No codebase yet?

Похожие видео

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.
1:11

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

The Humanoid Hub

25,125 просмотров • 7 месяцев назад

Meet Meter Command. A generative UI product for getting information, taking action, and building real-time software – all in natural language.
1:29

Meet Meter Command. A generative UI product for getting information, taking action, and building real-time software – all in natural language.

Meter

332,187 просмотров • 1 год назад

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥
1:33

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥

Zhengyi “Zen” Luo

15,851 просмотров • 7 месяцев назад

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.
2:53

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

The Humanoid Hub

69,415 просмотров • 1 год назад

Introducing Figure 02, a humanoid robot capable of natural language conversations thanks to OpenAI. What do you think?
0:30

Introducing Figure 02, a humanoid robot capable of natural language conversations thanks to OpenAI. What do you think?

MIT CSAIL

67,522 просмотров • 1 год назад

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.
0:45

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

The Humanoid Hub

20,430 просмотров • 1 год назад

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
1:32

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

61,856 просмотров • 7 месяцев назад

Mercury Command is live. AI built directly into your Mercury account that completes financial work. Natural language, real actions, and you approve everything. This is how banking* works now.
0:24

Mercury Command is live. AI built directly into your Mercury account that completes financial work. Natural language, real actions, and you approve everything. This is how banking* works now.

Mercury

141,722 просмотров • 5 дней назад

LMDrive Closed-Loop End-to-End Driving with LLM An end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions
1:40

LMDrive Closed-Loop End-to-End Driving with LLM An end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions

rsasaki0109

10,085 просмотров • 2 лет назад

LimX Dynamics unveiled COSA, a unified brain and body OS for humanoid robots. It lets their OLI robot think and act in real time from walking to manipulating objects using natural language. Cognition and control, fused.
1:18

LimX Dynamics unveiled COSA, a unified brain and body OS for humanoid robots. It lets their OLI robot think and act in real time from walking to manipulating objects using natural language. Cognition and control, fused.

XMAQUINA

15,054 просмотров • 5 месяцев назад

Big news from the TeleAI embodied intelligence team! They unveiled TextOp, a general cerebellar framework for real-time, text-driven humanoid control. TextOp lets humans control the robot via natural language, dynamically modifying instructions during runtime to instantly generate smooth, whole-body actions. This enables precise and highly versatile control. The framework uses a two-layer architecture: an action diffusion model (High-Level) and a general motion tracking policy (Low-Level). It’s a new paradigm that eliminates the need for pre-recorded scripts or manual programming.
1:20

Big news from the TeleAI embodied intelligence team! They unveiled TextOp, a general cerebellar framework for real-time, text-driven humanoid control. TextOp lets humans control the robot via natural language, dynamically modifying instructions during runtime to instantly generate smooth, whole-body actions. This enables precise and highly versatile control. The framework uses a two-layer architecture: an action diffusion model (High-Level) and a general motion tracking policy (Low-Level). It’s a new paradigm that eliminates the need for pre-recorded scripts or manual programming.

RoboHub🤖

23,804 просмотров • 7 месяцев назад

🚨 Your test cases shouldn’t break every time your UI changes. KaneAI writes, runs, and self-heals end-to-end tests directly from natural language built for real QA teams, not toy demos. From JIRA tickets to production-ready test code.
1:25

🚨 Your test cases shouldn’t break every time your UI changes. KaneAI writes, runs, and self-heals end-to-end tests directly from natural language built for real QA teams, not toy demos. From JIRA tickets to production-ready test code.

Darshal Jaitwar

70,504 просмотров • 4 месяцев назад

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation
0:27

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

105,428 просмотров • 8 месяцев назад

Today we’re releasing GitHub Spark — a new tool in Copilot that turns your ideas into full-stack apps, entirely in natural language.
1:21

Today we’re releasing GitHub Spark — a new tool in Copilot that turns your ideas into full-stack apps, entirely in natural language.

Satya Nadella

3,172,127 просмотров • 11 месяцев назад

Introducing Comet Shortcuts You can now automate repetitive boring web workflows with simple natural language prompts and access them with a /command anywhere We envision a future where people will be able to share and monetize custom shortcuts they built. More on that soon.
1:16

Introducing Comet Shortcuts You can now automate repetitive boring web workflows with simple natural language prompts and access them with a /command anywhere We envision a future where people will be able to share and monetize custom shortcuts they built. More on that soon.

Aravind Srinivas

164,515 просмотров • 10 месяцев назад

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.
2:39

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.

The Humanoid Hub

31,445 просмотров • 7 месяцев назад

Reading research papers shouldn't feel like learning a new language Introducing real-time AI voice chat for arXiv papers 🎙️ Talk directly about any research paper in any language. Ask questions and request explanations, all through natural conversation 🚀
1:43

Reading research papers shouldn't feel like learning a new language Introducing real-time AI voice chat for arXiv papers 🎙️ Talk directly about any research paper in any language. Ask questions and request explanations, all through natural conversation 🚀

alphaXiv

10,473 просмотров • 1 год назад

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:
3:05

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Zhengyi “Zen” Luo

92,104 просмотров • 1 год назад

‘CLONE’ – whole-body teleoperation of a humanoid. Intuitive control signals are captured by tracking the teleoperator’s head and hand poses using Apple Vision Pro. A Mixture-of-Experts policy takes the sparse input and synthesizes the corresponding whole-body humanoid pose.
1:08

‘CLONE’ – whole-body teleoperation of a humanoid. Intuitive control signals are captured by tracking the teleoperator’s head and hand poses using Apple Vision Pro. A Mixture-of-Experts policy takes the sparse input and synthesizes the corresponding whole-body humanoid pose.

The Humanoid Hub

38,155 просмотров • 1 год назад