正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

CMU Vision-Language-Autonomy update: The team just released SORT3D, the first general spatial relation toolbox for autonomous vision-language navigation that is fully integrated into real-robot systems! 🤖👀 Simulation and real-robot data is provided!:

CMU Robotics Institute

19,827 subscribers

18,234 次观看 • 1 年前 •via X (Twitter)

艺术科学技术教育

Anya Rossi• Live Now

Private livecam show

4 条评论

SaoAI 的头像

SaoAI1 年前

cool

MightyBot 的头像

MightyBot1 年前

🧠 Unified Search. Smarter Meetings. Effortless CRM. MightyBot is your AI agent platform for seamless workflows—record meetings, automate CRM updates, and find answers across apps in seconds. 🌟 Focus on what matters. We'll handle the grind.

Leila Anime 的头像

Leila Anime1 年前

Impressive innovation! SORT3D is pushing the boundaries of vision-language navigation, making autonomous robots smarter. Kudos to the CMU Robotics team! 🤖

LECCA Intern (Ø,G) 🍜 的头像

LECCA Intern (Ø,G) 🍜1 年前

The real-world applications of SORT3D are crazy. Home robots, warehouses, even search-and-rescue just leveled up.

相关视频

As part of the ongoing Vision-Language-Autonomy project, CMU researchers have developed a general navigation platform with an onboard autonomy stack to support Vision-Language-Navigation. It functions in simulation and on real robots 🤖💥 Check it out!

As part of the ongoing Vision-Language-Autonomy project, CMU researchers have developed a general navigation platform with an onboard autonomy stack to support Vision-Language-Navigation. It functions in simulation and on real robots 🤖💥 Check it out!

CMU Robotics Institute

23,706 次观看 • 1 年前

Qwen-VLA feels like one of the first real robotics foundation models. A single system trained across robot manipulation, navigation, egocentric human video, simulation, and vision-language reasoning instead of isolated robot policies.

Qwen-VLA feels like one of the first real robotics foundation models. A single system trained across robot manipulation, navigation, egocentric human video, simulation, and vision-language reasoning instead of isolated robot policies.

Robots Digest 🤖

14,601 次观看 • 1 个月前

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:

Danny Driess

1,272,497 次观看 • 3 年前

🚨CMU Vision-Language-Autonomy update: The team released a video to "find the refrigerator in the lounge"–– they are looking for new PhD & Master's students to work on long-horizon navigation & instruction! Contact Ji Zhang for more information:

🚨CMU Vision-Language-Autonomy update: The team released a video to "find the refrigerator in the lounge"–– they are looking for new PhD & Master's students to work on long-horizon navigation & instruction! Contact Ji Zhang for more information:

CMU Robotics Institute

20,362 次观看 • 9 个月前

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

NVIDIA Robotics

13,421 次观看 • 5 个月前

Physical Intelligence's π₀, a general-purpose robot foundation model that combines Internet-scale vision-language pretraining with robot interaction data to execute tasks. They aim "to develop foundation models that can control any robot to perform any task" Autonomous demos:

Physical Intelligence's π₀, a general-purpose robot foundation model that combines Internet-scale vision-language pretraining with robot interaction data to execute tasks. They aim "to develop foundation models that can control any robot to perform any task" Autonomous demos:

The Humanoid Hub

196,055 次观看 • 1 年前

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

Siyuan Huang

22,291 次观看 • 2 年前

QACE Dynamics | Autonomous Navigation Preview Today we are showing autonomous navigation running live inside QACE, the robot plans its route on the map, then adjusts in real time as lidar and vision pick up new obstacles. All of this runs through a single navigation block inside QACE, ready to plug into different robots without touching low-level control. This navigation block is one of the core building pieces of QACE autonomy, and it’s just the first of many we will be rolling out.

QACE Dynamics | Autonomous Navigation Preview Today we are showing autonomous navigation running live inside QACE, the robot plans its route on the map, then adjusts in real time as lidar and vision pick up new obstacles. All of this runs through a single navigation block inside QACE, ready to plug into different robots without touching low-level control. This navigation block is one of the core building pieces of QACE autonomy, and it’s just the first of many we will be rolling out.

Qace Dynamics

12,654 次观看 • 7 个月前

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

95,044 次观看 • 11 个月前

Check out this Circulus Pion robot at Computex 2026. It’s running on a Panther Lake processor, using P-cores for real-time control, E-cores to collect sensor data, the GPU for vision-language-action models, and the NPU for continuous perception.

Check out this Circulus Pion robot at Computex 2026. It’s running on a Panther Lake processor, using P-cores for real-time control, E-cores to collect sensor data, the GPU for vision-language-action models, and the NPU for continuous perception.

Intel Technology

10,752 次观看 • 24 天前

Introducing TraceVLA: a fully open-source Vision-Language-Action model reimagining spatial-temporal awareness: ✨ 3.5x gains on real robots, SOTA in simulation 💡 Fine-tunes on just 150K trajectories ⚡ Compact 4B model = 7B performance

Introducing TraceVLA: a fully open-source Vision-Language-Action model reimagining spatial-temporal awareness: ✨ 3.5x gains on real robots, SOTA in simulation 💡 Fine-tunes on just 150K trajectories ⚡ Compact 4B model = 7B performance

Yongyuan Liang

39,500 次观看 • 1 年前

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 次观看 • 1 年前

Excited to announce Tau Robotics (Tau Robotics). We are building a general AI for robots. We start by building millions of robot arms that learn in the real world. In the video, two robot arms are fully autonomous and controlled by a single neural network conditioned on different language instructions (four axes and five axes robot arms). The other two arms are teleoperated. The entire hardware cost in the video is about $1400. The video is at 1.5x speed.

Excited to announce Tau Robotics (Tau Robotics). We are building a general AI for robots. We start by building millions of robot arms that learn in the real world. In the video, two robot arms are fully autonomous and controlled by a single neural network conditioned on different language instructions (four axes and five axes robot arms). The other two arms are teleoperated. The entire hardware cost in the video is about $1400. The video is at 1.5x speed.

Alexander Koch

437,797 次观看 • 2 年前

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Physical Intelligence

116,845 次观看 • 1 年前

Developing general-purpose robot models is costly due to the need for extensive real-world data. 🤖 #NVIDIAIsaac GR00T addresses these challenges by providing #humanoidrobot developers with robot foundation models & more. #CES2025 📹

Developing general-purpose robot models is costly due to the need for extensive real-world data. 🤖 #NVIDIAIsaac GR00T addresses these challenges by providing #humanoidrobot developers with robot foundation models & more. #CES2025 📹

NVIDIA Robotics

19,055 次观看 • 1 年前

POV: you’re a delivery robot navigating the real world 🤖 We’ll call it Dot. Real streets. Real traffic. Real decisions happening in milliseconds. Dot’s autonomy stack was designed for real-world uncertainty. This is DoorDash Labs autonomy in action.

POV: you’re a delivery robot navigating the real world 🤖 We’ll call it Dot. Real streets. Real traffic. Real decisions happening in milliseconds. Dot’s autonomy stack was designed for real-world uncertainty. This is DoorDash Labs autonomy in action.

DoorDash

129,505 次观看 • 3 个月前

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N

Russell Mendonca

38,454 次观看 • 1 年前

Launching Dimensional: the operating system for physical space Command any robot in natural language, with universal abilities like spatial memory, autonomous navigation, manipulation, person tracking, more. Open source. Single pip install It’s time to program atoms, not bits

Launching Dimensional: the operating system for physical space Command any robot in natural language, with universal abilities like spatial memory, autonomous navigation, manipulation, person tracking, more. Open source. Single pip install It’s time to program atoms, not bits

stash

251,136 次观看 • 8 个月前

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

Siddhant Haldar

19,818 次观看 • 4 个月前