Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Command humanoids *directly* with natural language? Introducing LangWBC, a generative, end-to-end policy that turns natural language into real-world whole-body humanoid control! 💬→🦿Smooth, robust, surprisingly intuitive! See more 👉 #RSS2025

ZhongyuLi4's profile picture

Zhongyu Li

1,497 subscribers

14,142 görüntüleme • 1 yıl önce •via X (Twitter)

Bilim & TeknolojiEğitim#RSS2025
LIVE

Anya Rossi• Live Now

Private livecam show

9 Yorum

Zhongyu Li profil fotoğrafı
Zhongyu Li1 yıl önce

Intuition of LangWBC: Map language to whole-body motion, using a CVAE for its structured latent space→Enables generalization to linguistic variations & unseen latents! 1. Train lang.-agnostic RL experts for motion tracking 2. Distill into a single e2e policy conditioned on lang.

Zhongyu Li profil fotoğrafı
Zhongyu Li1 yıl önce

Real-world deployment shows LangWBC in action: ✅ Agile, versatile whole-body motion ✅ Smooth transitions—e.g., walking ➡️ running ✅ Emergent motions via latent space blending ("walk" + "sidestep" = diagonal!) ✅ Easy integration with LLMs for natural interaction

Zhongyu Li profil fotoğrafı
Zhongyu Li1 yıl önce

📄 This work has been accepted to #RSS2025! Check out the full paper here: Big thanks to our collaborators @Shaoyy2003 @x_h_ucb @qiayuanliao @t_k_233 @KoushilSreenath and everyone who supported this work!

TXYZ profil fotoğrafı
TXYZ1 yıl önce

⏰ Countdown alert! 🚀 Get free access to our new AI Writing feature before Dec 1st! Don't miss your chance to explore its power. Try it now—no cost, no catch! 🔗

LECCA Intern (Ø,G) 🍜 profil fotoğrafı
LECCA Intern (Ø,G) 🍜1 yıl önce

Language-to-motion in one shot. LangWBC feels like GPT for humanoids.

Sara Orbgirl profil fotoğrafı
Sara Orbgirl1 yıl önce

Imagine commanding robots with words, like a sci-fi dream come true! 🌠 Who knew talking could create motion? Impressive work! 🤖

Shi Soul profil fotoğrafı
Shi Soul1 yıl önce

seems the feasible motions are too few( walk, run, wave) can we do sth more?

Weishuai Zeng profil fotoğrafı
Weishuai Zeng1 yıl önce

Nice work Zhongyu! Congratulations~ May I know how do you obtain base linear velocity in real world as you mention in your paper? QwQ

Sanjar profil fotoğrafı
Sanjar1 yıl önce

Great work! No codebase yet?

Benzer Videolar

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.
1:11

TextOp: A real-time text-to-motion framework for humanoid robots. It allows users to instruct the robot using natural language and to modify commands on the fly, producing smooth, whole-body motions instantly.

The Humanoid Hub

25,125 görüntüleme • 7 ay önce

Meet Meter Command. A generative UI product for getting information, taking action, and building real-time software – all in natural language.
1:29

Meet Meter Command. A generative UI product for getting information, taking action, and building real-time software – all in natural language.

Meter

332,187 görüntüleme • 1 yıl önce

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥
1:33

Humanoids need a single, generalist control policy for all of their physical tasks, not a new one for every new chore or demo. A policy for walking can't dance. A policy for dancing can't support mowing the lawn. We need to scale up humanoid control for diverse behaviors, just like we did with language and video models. Watch this humanoid, supported by a single control policy, to start its day, mow the lawn, jog, dance, and even hold the elevator for a colleague. This is the "System 1" for humanoids—a robust, natural humanoid controller that will support general-purpose autonomy. Project Page: #Robotics #AI #NVIDIA #SONIC #FoundationModel 🤖🔥

Zhengyi “Zen” Luo

15,851 görüntüleme • 7 ay önce

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.
2:53

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

The Humanoid Hub

69,415 görüntüleme • 1 yıl önce

Introducing Figure 02, a humanoid robot capable of natural language conversations thanks to OpenAI. What do you think?
0:30

Introducing Figure 02, a humanoid robot capable of natural language conversations thanks to OpenAI. What do you think?

MIT CSAIL

67,522 görüntüleme • 1 yıl önce

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.
0:45

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

The Humanoid Hub

20,430 görüntüleme • 1 yıl önce

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
1:32

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

61,856 görüntüleme • 7 ay önce

Mercury Command is live. AI built directly into your Mercury account that completes financial work. Natural language, real actions, and you approve everything. This is how banking* works now.
0:24

Mercury Command is live. AI built directly into your Mercury account that completes financial work. Natural language, real actions, and you approve everything. This is how banking* works now.

Mercury

141,481 görüntüleme • 4 gün önce

LMDrive Closed-Loop End-to-End Driving with LLM An end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions
1:40

LMDrive Closed-Loop End-to-End Driving with LLM An end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions

rsasaki0109

10,085 görüntüleme • 2 yıl önce

LimX Dynamics unveiled COSA, a unified brain and body OS for humanoid robots. It lets their OLI robot think and act in real time from walking to manipulating objects using natural language. Cognition and control, fused.
1:18

LimX Dynamics unveiled COSA, a unified brain and body OS for humanoid robots. It lets their OLI robot think and act in real time from walking to manipulating objects using natural language. Cognition and control, fused.

XMAQUINA

15,054 görüntüleme • 5 ay önce

Big news from the TeleAI embodied intelligence team! They unveiled TextOp, a general cerebellar framework for real-time, text-driven humanoid control. TextOp lets humans control the robot via natural language, dynamically modifying instructions during runtime to instantly generate smooth, whole-body actions. This enables precise and highly versatile control. The framework uses a two-layer architecture: an action diffusion model (High-Level) and a general motion tracking policy (Low-Level). It’s a new paradigm that eliminates the need for pre-recorded scripts or manual programming.
1:20

Big news from the TeleAI embodied intelligence team! They unveiled TextOp, a general cerebellar framework for real-time, text-driven humanoid control. TextOp lets humans control the robot via natural language, dynamically modifying instructions during runtime to instantly generate smooth, whole-body actions. This enables precise and highly versatile control. The framework uses a two-layer architecture: an action diffusion model (High-Level) and a general motion tracking policy (Low-Level). It’s a new paradigm that eliminates the need for pre-recorded scripts or manual programming.

RoboHub🤖

23,804 görüntüleme • 7 ay önce

🚨 Your test cases shouldn’t break every time your UI changes. KaneAI writes, runs, and self-heals end-to-end tests directly from natural language built for real QA teams, not toy demos. From JIRA tickets to production-ready test code.
1:25

🚨 Your test cases shouldn’t break every time your UI changes. KaneAI writes, runs, and self-heals end-to-end tests directly from natural language built for real QA teams, not toy demos. From JIRA tickets to production-ready test code.

Darshal Jaitwar

70,504 görüntüleme • 4 ay önce

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation
0:27

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

105,428 görüntüleme • 8 ay önce

Today we’re releasing GitHub Spark — a new tool in Copilot that turns your ideas into full-stack apps, entirely in natural language.
1:21

Today we’re releasing GitHub Spark — a new tool in Copilot that turns your ideas into full-stack apps, entirely in natural language.

Satya Nadella

3,172,110 görüntüleme • 11 ay önce

Introducing Comet Shortcuts You can now automate repetitive boring web workflows with simple natural language prompts and access them with a /command anywhere We envision a future where people will be able to share and monetize custom shortcuts they built. More on that soon.
1:16

Introducing Comet Shortcuts You can now automate repetitive boring web workflows with simple natural language prompts and access them with a /command anywhere We envision a future where people will be able to share and monetize custom shortcuts they built. More on that soon.

Aravind Srinivas

164,515 görüntüleme • 10 ay önce

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.
2:39

NVIDIA researchers present SONIC, a generalist humanoid controller: It scales motion tracking on a single policy to achieve natural, robust whole-body movement. The scalable foundation avoids manual reward engineering and features a universal token space and kinematic planner to seamlessly integrate diverse inputs, including VR/video teleoperation and VLA foundation models, for real-world tasks.

The Humanoid Hub

31,445 görüntüleme • 7 ay önce

Reading research papers shouldn't feel like learning a new language Introducing real-time AI voice chat for arXiv papers 🎙️ Talk directly about any research paper in any language. Ask questions and request explanations, all through natural conversation 🚀
1:43

Reading research papers shouldn't feel like learning a new language Introducing real-time AI voice chat for arXiv papers 🎙️ Talk directly about any research paper in any language. Ask questions and request explanations, all through natural conversation 🚀

alphaXiv

10,473 görüntüleme • 1 yıl önce

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:
3:05

Introducing Omnigrasp: Grasping Diverse Objects with Simulated Humanoids. With Omnigrasp, we show that we can control a humanoid equipped with dexterous hands to grasp diverse objects (>1200) and follow diverse trajectories, with one policy! 🌐: 📜:

Zhengyi “Zen” Luo

92,104 görüntüleme • 1 yıl önce

‘CLONE’ – whole-body teleoperation of a humanoid. Intuitive control signals are captured by tracking the teleoperator’s head and hand poses using Apple Vision Pro. A Mixture-of-Experts policy takes the sparse input and synthesizes the corresponding whole-body humanoid pose.
1:08

‘CLONE’ – whole-body teleoperation of a humanoid. Intuitive control signals are captured by tracking the teleoperator’s head and hand poses using Apple Vision Pro. A Mixture-of-Experts policy takes the sparse input and synthesizes the corresponding whole-body humanoid pose.

The Humanoid Hub

38,155 görüntüleme • 1 yıl önce