Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

CMU Vision-Language-Autonomy update: The team just released SORT3D, the first general spatial relation toolbox for autonomous vision-language navigation that is fully integrated into real-robot systems! 🤖👀 Simulation and real-robot data is provided!:

CMU_Robotics's profile picture

CMU Robotics Institute

19,827 subscribers

18,234 Aufrufe • vor 1 Jahr •via X (Twitter)

KunstWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

4 Kommentare

Profilbild von SaoAI
SaoAIvor 1 Jahr

cool

Profilbild von MightyBot
MightyBotvor 1 Jahr

🧠 Unified Search. Smarter Meetings. Effortless CRM. MightyBot is your AI agent platform for seamless workflows—record meetings, automate CRM updates, and find answers across apps in seconds. 🌟 Focus on what matters. We'll handle the grind.

Profilbild von Leila Anime
Leila Animevor 1 Jahr

Impressive innovation! SORT3D is pushing the boundaries of vision-language navigation, making autonomous robots smarter. Kudos to the CMU Robotics team! 🤖

Profilbild von LECCA Intern (Ø,G) 🍜
LECCA Intern (Ø,G) 🍜vor 1 Jahr

The real-world applications of SORT3D are crazy. Home robots, warehouses, even search-and-rescue just leveled up.

Ähnliche Videos

As part of the ongoing Vision-Language-Autonomy project, CMU researchers have developed a general navigation platform with an onboard autonomy stack to support Vision-Language-Navigation. It functions in simulation and on real robots 🤖💥 Check it out!
0:29

As part of the ongoing Vision-Language-Autonomy project, CMU researchers have developed a general navigation platform with an onboard autonomy stack to support Vision-Language-Navigation. It functions in simulation and on real robots 🤖💥 Check it out!

CMU Robotics Institute

23,706 Aufrufe • vor 1 Jahr

Qwen-VLA feels like one of the first real robotics foundation models. A single system trained across robot manipulation, navigation, egocentric human video, simulation, and vision-language reasoning instead of isolated robot policies.
2:52

Qwen-VLA feels like one of the first real robotics foundation models. A single system trained across robot manipulation, navigation, egocentric human video, simulation, and vision-language reasoning instead of isolated robot policies.

Robots Digest 🤖

14,519 Aufrufe • vor 11 Tagen

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:
0:42

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:

Danny Driess

1,272,447 Aufrufe • vor 3 Jahren

🚨CMU Vision-Language-Autonomy update: The team released a video to "find the refrigerator in the lounge"–– they are looking for new PhD & Master's students to work on long-horizon navigation & instruction! Contact Ji Zhang for more information:
1:00

🚨CMU Vision-Language-Autonomy update: The team released a video to "find the refrigerator in the lounge"–– they are looking for new PhD & Master's students to work on long-horizon navigation & instruction! Contact Ji Zhang for more information:

CMU Robotics Institute

20,279 Aufrufe • vor 9 Monaten

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗
0:29

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

NVIDIA Robotics

13,390 Aufrufe • vor 4 Monaten

Physical Intelligence's π₀, a general-purpose robot foundation model that combines Internet-scale vision-language pretraining with robot interaction data to execute tasks. They aim "to develop foundation models that can control any robot to perform any task" Autonomous demos:
3:36

Physical Intelligence's π₀, a general-purpose robot foundation model that combines Internet-scale vision-language pretraining with robot interaction data to execute tasks. They aim "to develop foundation models that can control any robot to perform any task" Autonomous demos:

The Humanoid Hub

196,055 Aufrufe • vor 1 Jahr

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:
0:58

🎤🎤 Excited to introduce COME-robot🤖🤖, Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V. It is the first closed-loop framework utilizing the vision-language foundation model for open-ended reasoning and adaptive planning in real-world scenarios. COME-robot demonstrates a significant improvement in task success rate (~25%) compared to SOTA methods. Project: Arxiv:

Siyuan Huang

22,291 Aufrufe • vor 2 Jahren

QACE Dynamics | Autonomous Navigation Preview Today we are showing autonomous navigation running live inside QACE, the robot plans its route on the map, then adjusts in real time as lidar and vision pick up new obstacles. All of this runs through a single navigation block inside QACE, ready to plug into different robots without touching low-level control. This navigation block is one of the core building pieces of QACE autonomy, and it’s just the first of many we will be rolling out.
0:58

QACE Dynamics | Autonomous Navigation Preview Today we are showing autonomous navigation running live inside QACE, the robot plans its route on the map, then adjusts in real time as lidar and vision pick up new obstacles. All of this runs through a single navigation block inside QACE, ready to plug into different robots without touching low-level control. This navigation block is one of the core building pieces of QACE autonomy, and it’s just the first of many we will be rolling out.

Qace Dynamics

12,654 Aufrufe • vor 6 Monaten

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:
3:55

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

94,717 Aufrufe • vor 11 Monaten

Introducing TraceVLA: a fully open-source Vision-Language-Action model reimagining spatial-temporal awareness: ✨ 3.5x gains on real robots, SOTA in simulation 💡 Fine-tunes on just 150K trajectories ⚡ Compact 4B model = 7B performance
0:29

Introducing TraceVLA: a fully open-source Vision-Language-Action model reimagining spatial-temporal awareness: ✨ 3.5x gains on real robots, SOTA in simulation 💡 Fine-tunes on just 150K trajectories ⚡ Compact 4B model = 7B performance

Yongyuan Liang

39,378 Aufrufe • vor 1 Jahr

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)
0:33

A big part of scaling robot learning to solve real-world problems is that we somehow need to get enough diverse, high-quality data to train our robots to perform useful things. GPT and its fellow large language models were bootstrapped and proved out on a massive dataset of real-world language data. Unfortunately, despite our best efforts, similarly massive datasets don’t really exist for robotics — so, in our unending pursuit of high-quality, useful data, we turn to simulation. I compared a couple recent works on sim-to-real robot manipulation, which discuss how to train perception-driven manipulation policies in simulation, in such a way that they’re useful in the real world. - DextraH-RGB, from NVIDIA - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation, also from NVIDIA — specifically the GEAR lab - Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids, another GEAR lab paper - Local Policies Enable Zero-shot Long-Horizon Manipulation, from CMU (video from DextrAH-RGB)

Chris Paxton

20,486 Aufrufe • vor 1 Jahr

Excited to announce Tau Robotics (Tau Robotics). We are building a general AI for robots. We start by building millions of robot arms that learn in the real world. In the video, two robot arms are fully autonomous and controlled by a single neural network conditioned on different language instructions (four axes and five axes robot arms). The other two arms are teleoperated. The entire hardware cost in the video is about $1400. The video is at 1.5x speed.
0:39

Excited to announce Tau Robotics (Tau Robotics). We are building a general AI for robots. We start by building millions of robot arms that learn in the real world. In the video, two robot arms are fully autonomous and controlled by a single neural network conditioned on different language instructions (four axes and five axes robot arms). The other two arms are teleoperated. The entire hardware cost in the video is about $1400. The video is at 1.5x speed.

Alexander Koch

437,780 Aufrufe • vor 2 Jahren

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).
3:10

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Physical Intelligence

116,845 Aufrufe • vor 1 Jahr

Developing general-purpose robot models is costly due to the need for extensive real-world data. 🤖 #NVIDIAIsaac GR00T addresses these challenges by providing #humanoidrobot developers with robot foundation models & more. #CES2025 📹
0:30

Developing general-purpose robot models is costly due to the need for extensive real-world data. 🤖 #NVIDIAIsaac GR00T addresses these challenges by providing #humanoidrobot developers with robot foundation models & more. #CES2025 📹

NVIDIA Robotics

19,055 Aufrufe • vor 1 Jahr

POV: you’re a delivery robot navigating the real world 🤖 We’ll call it Dot. Real streets. Real traffic. Real decisions happening in milliseconds. Dot’s autonomy stack was designed for real-world uncertainty. This is DoorDash Labs autonomy in action.
0:49

POV: you’re a delivery robot navigating the real world 🤖 We’ll call it Dot. Real streets. Real traffic. Real decisions happening in milliseconds. Dot’s autonomy stack was designed for real-world uncertainty. This is DoorDash Labs autonomy in action.

DoorDash

129,420 Aufrufe • vor 2 Monaten

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N
1:34

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N

Russell Mendonca

38,454 Aufrufe • vor 1 Jahr

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.
0:50

Robot foundation models are limited by costly real data, while simulation data is plentiful but visually mismatched to reality. We present Point Bridge, a method that enables zero-shot sim-to-real transfer for robot learning with minimal visual alignment.

Siddhant Haldar

19,730 Aufrufe • vor 3 Monaten

Launching Dimensional: the operating system for physical space Command any robot in natural language, with universal abilities like spatial memory, autonomous navigation, manipulation, person tracking, more. Open source. Single pip install It’s time to program atoms, not bits
1:52

Launching Dimensional: the operating system for physical space Command any robot in natural language, with universal abilities like spatial memory, autonomous navigation, manipulation, person tracking, more. Open source. Single pip install It’s time to program atoms, not bits

stash

251,136 Aufrufe • vor 7 Monaten

Robot tried Apple Vision Pro for the first time.
0:22

Robot tried Apple Vision Pro for the first time.

Brian Roemmele

246,245 Aufrufe • vor 1 Jahr