Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Our Robotic Foundation Model, trained on data from real-world warehouse operations and simulated scenarios, enables our robots to identify objects, understand 3D spaces, and predict optimal grasping and placement.

Covariant

11,610 subscribers

33,508 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

7 Comments

Altiam Kabir1 year ago

Impressive innovation! Your robots are revolutionizing warehouse operations.

Rish1 year ago

@PhysicalAI @ipoupyrev

atharva1 year ago

extremely cool

Michael Spencer1 year ago

So where's the tweet about the Amazon deal already?

𝗖𝗵𝗿𝗼𝗻𝗼𝗻𝗮𝘂𝘁1 year ago

⚡

ObotX - Open Source Robotics1 year ago

Amazing! Maybe you can simulate on such a construction?

Anthara Fairooz1 year ago

Impressive tech advancements! Exciting potential ahead.

Related Videos

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

The Humanoid Hub

30,148 views • 9 months ago

TidyBot: Personalized Robot Assistance with Large Language Models approach enables fast adaptation and achieves 91.2% accuracy on unseen objects in our benchmark dataset. We also demonstrate our approach on a real-world mobile manipulator called TidyBot, which successfully puts away 85.0% of objects in real-world test scenarios abs: project page: paper page:

TidyBot: Personalized Robot Assistance with Large Language Models approach enables fast adaptation and achieves 91.2% accuracy on unseen objects in our benchmark dataset. We also demonstrate our approach on a real-world mobile manipulator called TidyBot, which successfully puts away 85.0% of objects in real-world test scenarios abs: project page: paper page:

AK

325,986 views • 3 years ago

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Jianlan Luo

53,748 views • 14 days ago

Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained from real and simulated data which can perform a wide variety of manipulation tasks on a real robot, including grasping or handling articulated objects, all without any additional fine tuning. Wenlong Huang joins us to tell us more about what makes this work and how it’s different from other world models. Watch Episode #83 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained from real and simulated data which can perform a wide variety of manipulation tasks on a real robot, including grasping or handling articulated objects, all without any additional fine tuning. Wenlong Huang joins us to tell us more about what makes this work and how it’s different from other world models. Watch Episode #83 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

16,197 views • 16 days ago

In just ~3 months, as a solo founder with no prior robotics experience, General Trajectory trained a foundation model for dexterous manipulation that lets humanoid robots pick up unseen objects and perform real-world work. It generalizes to novel objects and scenes, including cases where prior SoTA models achieve 0% success. Congrats on the launch Joshua!

In just ~3 months, as a solo founder with no prior robotics experience, General Trajectory trained a foundation model for dexterous manipulation that lets humanoid robots pick up unseen objects and perform real-world work. It generalizes to novel objects and scenes, including cases where prior SoTA models achieve 0% success. Congrats on the launch Joshua!

Y Combinator

68,505 views • 5 months ago

Adobe is entering the image-to-3D game! Their new method, LRM, can create high-fidelity 3D meshes from a single image in just 5 seconds 🔥 It's trained on 1 million objects and is able to generate objects from real-world images and generative AI models.

Adobe is entering the image-to-3D game! Their new method, LRM, can create high-fidelity 3D meshes from a single image in just 5 seconds 🔥 It's trained on 1 million objects and is able to generate objects from real-world images and generative AI models.

Dreaming Tulpa 🥓👑

270,918 views • 2 years ago

In our newest Tech Talks episode, Roblox CEO & Founder David Baszucki, Kiran Bhat, and Alberto Hojel discuss what makes Roblox's 3D-native foundation model different from your standard video model, allowing creators to build functional objects and worlds from prompts. Watch the full episode on YouTube now.

In our newest Tech Talks episode, Roblox CEO & Founder David Baszucki, Kiran Bhat, and Alberto Hojel discuss what makes Roblox's 3D-native foundation model different from your standard video model, allowing creators to build functional objects and worlds from prompts. Watch the full episode on YouTube now.

Roblox

125,251 views • 3 months ago

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,780 views • 2 months ago

Our CTO boyuan is demoing our new localization algorithm, which surpasses leading industry solutions. Localization enables a robot to determine its position in an environment, which is essential for navigation. While most systems require robots to start from a predefined location, ours doesn’t. It lets robots boot up anywhere and immediately locate themselves, making deployment far more flexible for real world scenarios.

Our CTO boyuan is demoing our new localization algorithm, which surpasses leading industry solutions. Localization enables a robot to determine its position in an environment, which is essential for navigation. While most systems require robots to start from a predefined location, ours doesn’t. It lets robots boot up anywhere and immediately locate themselves, making deployment far more flexible for real world scenarios.

OpenMind

108,390 views • 6 months ago

Sharing a few more demos of our first #LargeBehaviorModel (LBM) at TRI. 1/🍎This model enables a robot to learned to core and cut an apple into multiple slices autonomously. We trained our diffusion-based LBM on almost 1,700 hours of robot data, conducted 1,800 real-world evaluation rollouts, and ran over 47,000 simulation rollouts to rigorously study its capabilities. Learn more here:

Sharing a few more demos of our first #LargeBehaviorModel (LBM) at TRI. 1/🍎This model enables a robot to learned to core and cut an apple into multiple slices autonomously. We trained our diffusion-based LBM on almost 1,700 hours of robot data, conducted 1,800 real-world evaluation rollouts, and ran over 47,000 simulation rollouts to rigorously study its capabilities. Learn more here:

Zubair Irshad

12,202 views • 10 months ago

For large-scale robotic deployment🤖 in the real-world 🌏, robots must adapt to changes in environment and objects. Ever questioned the generalizability of your robot's manipulation policy? Put it to the test with The Colosseum 🏛️. Check out our project:

For large-scale robotic deployment🤖 in the real-world 🌏, robots must adapt to changes in environment and objects. Ever questioned the generalizability of your robot's manipulation policy? Put it to the test with The Colosseum 🏛️. Check out our project:

Jiafei Duan

36,617 views • 2 years ago

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,365 views • 2 years ago

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

AI at Meta

17,858 views • 6 months ago

🚀 First step to unlocking Generalist Robots! Introducing 🤖LAPA🤖, a new SOTA open-sourced 7B VLA pretrained without using action labels. 💪SOTA VLA trained with Open X (outperforming OpenVLA on cross and multi embodiment) 😯LAPA enables learning from human videos, unlocking potential for robotic foundation model ❗Over 30x pretraining efficiency for VLA training 🤗Code and checkpoints are all open-sourced!

🚀 First step to unlocking Generalist Robots! Introducing 🤖LAPA🤖, a new SOTA open-sourced 7B VLA pretrained without using action labels. 💪SOTA VLA trained with Open X (outperforming OpenVLA on cross and multi embodiment) 😯LAPA enables learning from human videos, unlocking potential for robotic foundation model ❗Over 30x pretraining efficiency for VLA training 🤗Code and checkpoints are all open-sourced!

Seonghyeon Ye

33,239 views • 1 year ago

Can we bring human-like Touch to robots🤖? Introducing our CoRL work on 3D-ViTac. Humans rely on both vision 👁️ and touch 🫳 for complex tasks. With combined visual-tactile sensing, robots can now tackle challenging tasks, like precise in-hand reorientation, fragile objects grasping. Website: #Robotics #CoRL2024 #Touch #tactile #AI #ML

Can we bring human-like Touch to robots🤖? Introducing our CoRL work on 3D-ViTac. Humans rely on both vision 👁️ and touch 🫳 for complex tasks. With combined visual-tactile sensing, robots can now tackle challenging tasks, like precise in-hand reorientation, fragile objects grasping. Website: #Robotics #CoRL2024 #Touch #tactile #AI #ML

Binghao Huang

49,500 views • 1 year ago

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

Jesse Zhang

99,814 views • 3 months ago

Introducing Meta Locate 3D: a model for accurate object localization in 3D environments. Learn how Meta Locate 3D can help robots accurately understand their surroundings and interact more naturally with humans. You can download the model and dataset, read our research paper, and even try a demo!

Introducing Meta Locate 3D: a model for accurate object localization in 3D environments. Learn how Meta Locate 3D can help robots accurately understand their surroundings and interact more naturally with humans. You can download the model and dataset, read our research paper, and even try a demo!

AI at Meta

81,287 views • 1 year ago

Drop the camera, and bang -- the robots do the job without camera calibration or additional data collection. We want robots to manipulate anywhere! Our new work, **Maniwhere**, enables robots to manipulate objects from any camera view with any background. Isn't this the generalization every roboticist is looking for? How do we achieve this with sim2real? You can check this thread from Zhecheng Yuan ! And the website is here:

Drop the camera, and bang -- the robots do the job without camera calibration or additional data collection. We want robots to manipulate anywhere! Our new work, Maniwhere, enables robots to manipulate objects from any camera view with any background. Isn't this the generalization every roboticist is looking for? How do we achieve this with sim2real? You can check this thread from Zhecheng Yuan ! And the website is here:

Huazhe Harry Xu

26,214 views • 1 year ago

🏗️ Policy Adaptation from Foundation Model Feedback #CVPR2023 Instead of using foundation model as a pre-trained encoder (generator), we use it as a Teacher (discriminator) to tell where our policy did wrong and helps it adapts to new envs and tasks.

🏗️ Policy Adaptation from Foundation Model Feedback #CVPR2023 Instead of using foundation model as a pre-trained encoder (generator), we use it as a Teacher (discriminator) to tell where our policy did wrong and helps it adapts to new envs and tasks.

Xiaolong Wang

24,396 views • 3 years ago