正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Command humanoids directly with natural language? Introducing LangWBC, a generative, end-to-end policy that turns natural language into real-world whole-body humanoid control! 💬→🦿Smooth, robust, surprisingly intuitive! See more 👉 #RSS2025

Zhongyu Li

1,473 subscribers

14,109 次观看 • 1 年前 •via X (Twitter)

科学技术教育 #RSS2025

Anya Rossi• Live Now

Private livecam show

9 条评论

Zhongyu Li 的头像

Zhongyu Li1 年前

Intuition of LangWBC: Map language to whole-body motion, using a CVAE for its structured latent space→Enables generalization to linguistic variations & unseen latents! 1. Train lang.-agnostic RL experts for motion tracking 2. Distill into a single e2e policy conditioned on lang.

Zhongyu Li 的头像

Zhongyu Li1 年前

Real-world deployment shows LangWBC in action: ✅ Agile, versatile whole-body motion ✅ Smooth transitions—e.g., walking ➡️ running ✅ Emergent motions via latent space blending ("walk" + "sidestep" = diagonal!) ✅ Easy integration with LLMs for natural interaction

Zhongyu Li 的头像

Zhongyu Li1 年前

📄 This work has been accepted to #RSS2025! Check out the full paper here: Big thanks to our collaborators @Shaoyy2003 @x_h_ucb @qiayuanliao @t_k_233 @KoushilSreenath and everyone who supported this work!

TXYZ 的头像

TXYZ1 年前

⏰ Countdown alert! 🚀 Get free access to our new AI Writing feature before Dec 1st! Don't miss your chance to explore its power. Try it now—no cost, no catch! 🔗

LECCA Intern (Ø,G) 🍜 的头像

LECCA Intern (Ø,G) 🍜1 年前

Language-to-motion in one shot. LangWBC feels like GPT for humanoids.

Sara Orbgirl 的头像

Sara Orbgirl1 年前

Imagine commanding robots with words, like a sci-fi dream come true! 🌠 Who knew talking could create motion? Impressive work! 🤖

Shi Soul 的头像

Shi Soul1 年前

seems the feasible motions are too few( walk, run, wave) can we do sth more?

Weishuai Zeng 的头像

Weishuai Zeng1 年前

Nice work Zhongyu! Congratulations~ May I know how do you obtain base linear velocity in real world as you mention in your paper? QwQ

Sanjar 的头像

Sanjar1 年前

Great work! No codebase yet?

相关视频

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

The Humanoid Hub

25,125 次观看 • 6 个月前

Meet Meter Command. A generative UI product for getting information, taking action, and building real-time software – all in natural language.

Meet Meter Command. A generative UI product for getting information, taking action, and building real-time software – all in natural language.

Meter

332,187 次观看 • 1 年前

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥

Zhengyi “Zen” Luo

15,849 次观看 • 7 个月前

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

The Humanoid Hub

69,402 次观看 • 1 年前

Introducing Figure 02, a humanoid robot capable of natural language conversations thanks to OpenAI. What do you think?

Introducing Figure 02, a humanoid robot capable of natural language conversations thanks to OpenAI. What do you think?

MIT CSAIL

67,522 次观看 • 1 年前

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

The Humanoid Hub

20,430 次观看 • 1 年前

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

61,761 次观看 • 7 个月前

LMDrive Closed-Loop End-to-End Driving with LLM An end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions

LMDrive Closed-Loop End-to-End Driving with LLM An end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions

rsasaki0109

10,085 次观看 • 2 年前

LimX Dynamics unveiled COSA, a unified brain and body OS for humanoid robots. It lets their OLI robot think and act in real time from walking to manipulating objects using natural language. Cognition and control, fused.

LimX Dynamics unveiled COSA, a unified brain and body OS for humanoid robots. It lets their OLI robot think and act in real time from walking to manipulating objects using natural language. Cognition and control, fused.

XMAQUINA

15,054 次观看 • 4 个月前

Big news from the TeleAI embodied intelligence team! They unveiled TextOp, a general cerebellar framework for real-time, text-driven humanoid control. TextOp lets humans control the robot via natural language, dynamically modifying instructions during runtime to instantly generate smooth, whole-body actions. This enables precise and highly versatile control. The framework uses a two-layer architecture: an action diffusion model (High-Level) and a general motion tracking policy (Low-Level). It’s a new paradigm that eliminates the need for pre-recorded scripts or manual programming.

Big news from the TeleAI embodied intelligence team! They unveiled TextOp, a general cerebellar framework for real-time, text-driven humanoid control. TextOp lets humans control the robot via natural language, dynamically modifying instructions during runtime to instantly generate smooth, whole-body actions. This enables precise and highly versatile control. The framework uses a two-layer architecture: an action diffusion model (High-Level) and a general motion tracking policy (Low-Level). It’s a new paradigm that eliminates the need for pre-recorded scripts or manual programming.

RoboHub🤖

23,804 次观看 • 6 个月前

🚨 Your test cases shouldn’t break every time your UI changes. KaneAI writes, runs, and self-heals end-to-end tests directly from natural language built for real QA teams, not toy demos. From JIRA tickets to production-ready test code.

🚨 Your test cases shouldn’t break every time your UI changes. KaneAI writes, runs, and self-heals end-to-end tests directly from natural language built for real QA teams, not toy demos. From JIRA tickets to production-ready test code.

Darshal Jaitwar

70,504 次观看 • 4 个月前

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

103,815 次观看 • 8 个月前

Today we’re releasing GitHub Spark — a new tool in Copilot that turns your ideas into full-stack apps, entirely in natural language.

Today we’re releasing GitHub Spark — a new tool in Copilot that turns your ideas into full-stack apps, entirely in natural language.

Satya Nadella

3,171,722 次观看 • 10 个月前

Introducing Comet Shortcuts You can now automate repetitive boring web workflows with simple natural language prompts and access them with a /command anywhere We envision a future where people will be able to share and monetize custom shortcuts they built. More on that soon.

Introducing Comet Shortcuts You can now automate repetitive boring web workflows with simple natural language prompts and access them with a /command anywhere We envision a future where people will be able to share and monetize custom shortcuts they built. More on that soon.

Aravind Srinivas

164,482 次观看 • 10 个月前

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.

The Humanoid Hub

31,445 次观看 • 6 个月前

Reading research papers shouldn't feel like learning a new language Introducing real-time AI voice chat for arXiv papers 🎙️ Talk directly about any research paper in any language. Ask questions and request explanations, all through natural conversation 🚀

Reading research papers shouldn't feel like learning a new language Introducing real-time AI voice chat for arXiv papers 🎙️ Talk directly about any research paper in any language. Ask questions and request explanations, all through natural conversation 🚀

alphaXiv

10,473 次观看 • 1 年前

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Zhengyi “Zen” Luo

92,104 次观看 • 1 年前

‘CLONE’ – whole-body teleoperation of a humanoid. Intuitive control signals are captured by tracking the teleoperator’s head and hand poses using Apple Vision Pro. A Mixture-of-Experts policy takes the sparse input and synthesizes the corresponding whole-body humanoid pose.

‘CLONE’ – whole-body teleoperation of a humanoid. Intuitive control signals are captured by tracking the teleoperator’s head and hand poses using Apple Vision Pro. A Mixture-of-Experts policy takes the sparse input and synthesizes the corresponding whole-body humanoid pose.

The Humanoid Hub

37,417 次观看 • 1 年前

Kinda makes sense that natural language is the final programming language?

Kinda makes sense that natural language is the final programming language?

Sara Dietschy 🍑y

15,333 次观看 • 8 个月前