Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We trained a robotic foundation model that can drive mobile robots in six different countries, and navigate Sproul Plaza in midday on the UC Berkeley campus! Some cool new work w/ noriaki_hirose, Lydia Ignatova, Kyle Stachowicz, Catherine Glossop, Dhruv Shah

Sergey Levine

130,388 subscribers

52,962 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

5 Comments

Michael Cho - Rbt/Acc1 year ago

@NoriakiHirose @KyleStachowicz @CatGlossop @shahdhruv_ Thanks for giving meaning to what we do @frodobots . Means a lot to us that our work help enable research like this; onwards!

Winnie1 year ago

@NoriakiHirose @KyleStachowicz @CatGlossop @shahdhruv_ Impressive! It took a lot of courage to write a PDF on failed methods.

paras1 year ago

@NoriakiHirose @KyleStachowicz @CatGlossop @shahdhruv_ Awesome work!!

Felipe Chávez1 year ago

@NoriakiHirose @KyleStachowicz @CatGlossop @shahdhruv_ Promising !

zhao1 year ago

@NoriakiHirose @KyleStachowicz @CatGlossop @shahdhruv_ dm

Related Videos

Antisemitism in the process of being normalized. Sproul plaza, UC Berkeley.

Antisemitism in the process of being normalized. Sproul plaza, UC Berkeley.

Shamik Dasgupta

29,029 views • 2 years ago

We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & Andrew Wagenmaker

We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & Andrew Wagenmaker

Sergey Levine

36,232 views • 16 days ago

Our Robotic Foundation Model, trained on data from real-world warehouse operations and simulated scenarios, enables our robots to identify objects, understand 3D spaces, and predict optimal grasping and placement.

Our Robotic Foundation Model, trained on data from real-world warehouse operations and simulated scenarios, enables our robots to identify objects, understand 3D spaces, and predict optimal grasping and placement.

Covariant

33,623 views • 1 year ago

.Basecamp Research trained EDEN, a 28B-parameter foundation model for biology, on Nebius: 1,024 GPUs, six weeks, no major interruptions. The result: new scaling laws for biology; new antibiotics and gene editors already in the lab.

.Basecamp Research trained EDEN, a 28B-parameter foundation model for biology, on Nebius: 1,024 GPUs, six weeks, no major interruptions. The result: new scaling laws for biology; new antibiotics and gene editors already in the lab.

Nebius

20,947 views • 3 days ago

What happens when vision🤝 robotics meet? Happy to share our new work on Pretraining Robotic Foundational Models!🔥 ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model. Berkeley AI Research😊

What happens when vision🤝 robotics meet? Happy to share our new work on Pretraining Robotic Foundational Models!🔥 ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model. Berkeley AI Research😊

Roei Herzig

62,711 views • 1 year ago

Diffusion models make great images. But can they drive robots? Usually that gets complicated really fast. We figured out how to get a Stable Diffusion model (based on Instruct pix2pix) to drive robotic instruction following. Simple recipe, works on a wide range of tasks. Thread👇

Diffusion models make great images. But can they drive robots? Usually that gets complicated really fast. We figured out how to get a Stable Diffusion model (based on Instruct pix2pix) to drive robotic instruction following. Simple recipe, works on a wide range of tasks. Thread👇

Sergey Levine

126,523 views • 2 years ago

Our San Francisco Bay Area office CAIR-SFBA today condemned the alleged assault on Malak Afaneh, a Palestinian Muslim University of California Berkeley (UC Berkeley) law student, by UC Berkeley Law professor Catherine Fisk. Zahra Billoo

Our San Francisco Bay Area office CAIR-SFBA today condemned the alleged assault on Malak Afaneh, a Palestinian Muslim University of California Berkeley (UC Berkeley) law student, by UC Berkeley Law professor Catherine Fisk. Zahra Billoo

CAIR National

1,405,193 views • 2 years ago

Today, we present a step-change in robotic AI Sunday. Introducing ACT-1: A frontier robot foundation model trained on zero robot data. - Ultra long-horizon tasks - Zero-shot generalization - Advanced dexterity 🧵->

Today, we present a step-change in robotic AI Sunday. Introducing ACT-1: A frontier robot foundation model trained on zero robot data. - Ultra long-horizon tasks - Zero-shot generalization - Advanced dexterity 🧵->

Tony Zhao

2,045,896 views • 7 months ago

🤖ROBOTS ARE GETTING SMARTER AT TOUCHING THE REAL WORLD Researchers from UC Berkeley, NVIDIA and Stanford introduced T-Rex, a framework that combines vision, language and tactile sensing. Instead of relying on cameras alone, robots can now respond to physical contact in real time. Robots are no longer just seeing objects. They’re learning how to FEEL them.

🤖ROBOTS ARE GETTING SMARTER AT TOUCHING THE REAL WORLD Researchers from UC Berkeley, NVIDIA and Stanford introduced T-Rex, a framework that combines vision, language and tactile sensing. Instead of relying on cameras alone, robots can now respond to physical contact in real time. Robots are no longer just seeing objects. They’re learning how to FEEL them.

Coin Bureau

27,388 views • 21 days ago

Robots help us do tasks in space that are too dangerous or challenging for humans. We’re working on a new generation of robotic spacecraft that can work together without being controlled by astronauts or teams on Earth:

Robots help us do tasks in space that are too dangerous or challenging for humans. We’re working on a new generation of robotic spacecraft that can work together without being controlled by astronauts or teams on Earth:

NASA

901,066 views • 2 years ago

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 views • 2 years ago

Speak with ChatGPT: You can now use voice to engage in a back-and-forth conversation with your assistant. The hyper-realistic text-to-speech model allows you to choose from five different voices. On mobile, opt-in to voice in Settings → New Features on the mobile app.

Speak with ChatGPT: You can now use voice to engage in a back-and-forth conversation with your assistant. The hyper-realistic text-to-speech model allows you to choose from five different voices. On mobile, opt-in to voice in Settings → New Features on the mobile app.

Rowan Cheung

55,777 views • 2 years ago

Last night at UC Berkeley, Jewish students were threatened, assaulted, and prevented from attending a speech by a Jewish speaker on campus. Campus police shut down this private event when it became clear that they could not protect the students. Multiple students reported being spat on, screamed at, called derogatory names like "dirty Jew" or "you Jew," and physically assaulted by protestors. Jewish students deserve to navigate their lives without fear, and it is crucial that there are significant repercussions for those hindering this goal, including the responsible organizations and individuals. Ultimately, the UC Berkeley administration bears the responsibility of ensuring the safety of Jewish students on campus, a duty that was not fulfilled last night.

Last night at UC Berkeley, Jewish students were threatened, assaulted, and prevented from attending a speech by a Jewish speaker on campus. Campus police shut down this private event when it became clear that they could not protect the students. Multiple students reported being spat on, screamed at, called derogatory names like "dirty Jew" or "you Jew," and physically assaulted by protestors. Jewish students deserve to navigate their lives without fear, and it is crucial that there are significant repercussions for those hindering this goal, including the responsible organizations and individuals. Ultimately, the UC Berkeley administration bears the responsibility of ensuring the safety of Jewish students on campus, a duty that was not fulfilled last night.

JCRC Bay Area

1,146,978 views • 2 years ago

Many of you asked for code & weights for π₀, we are happy to announce that we are releasing π₀ and pre-trained checkpoints in our new openpi repository! We tested the model on a few public robots, and we include code for you to fine-tune it yourself.

Many of you asked for code & weights for π₀, we are happy to announce that we are releasing π₀ and pre-trained checkpoints in our new openpi repository! We tested the model on a few public robots, and we include code for you to fine-tune it yourself.

Physical Intelligence

441,236 views • 1 year ago

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Jianlan Luo

63,226 views • 1 month ago

We hosted Prof. Alyosha Efros (UC Berkeley) at Skild AI! He didn't believe that robots could actually cook eggs reliably. :) Tested back-to-back 5times without fail! One batch of scrambled eggs every ~2.5mins nonstop. The same model assembles a GPU on a server rack too.

We hosted Prof. Alyosha Efros (UC Berkeley) at Skild AI! He didn't believe that robots could actually cook eggs reliably. :) Tested back-to-back 5times without fail! One batch of scrambled eggs every ~2.5mins nonstop. The same model assembles a GPU on a server rack too.

Deepak Pathak

174,585 views • 3 months ago

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

Chelsea Finn

264,399 views • 3 years ago

Humans learn and improve from failures. Similarly, foundation models adapt based on human feedback. Can we leverage this failure understanding to enhance robotics systems that use foundation models? Introducing AHA—a vision-language model for detecting and reasoning over failures in robotic manipulation. Project page: 🧵Thread👇 Aha!

Humans learn and improve from failures. Similarly, foundation models adapt based on human feedback. Can we leverage this failure understanding to enhance robotics systems that use foundation models? Introducing AHA—a vision-language model for detecting and reasoning over failures in robotic manipulation. Project page: 🧵Thread👇 Aha!

Jiafei Duan

48,773 views • 1 year ago

In just ~3 months, as a solo founder with no prior robotics experience, General Trajectory trained a foundation model for dexterous manipulation that lets humanoid robots pick up unseen objects and perform real-world work. It generalizes to novel objects and scenes, including cases where prior SoTA models achieve 0% success. Congrats on the launch Joshua!

In just ~3 months, as a solo founder with no prior robotics experience, General Trajectory trained a foundation model for dexterous manipulation that lets humanoid robots pick up unseen objects and perform real-world work. It generalizes to novel objects and scenes, including cases where prior SoTA models achieve 0% success. Congrats on the launch Joshua!

Y Combinator

68,556 views • 6 months ago