Sergey Levine's banner

Sergey Levine

@svlevine • 128,158 subscribers

Associate Professor at UC Berkeley Co-founder, Physical Intelligence

Shorts

Tmrw(Sat) I'll talk about pi-zero & HIL-SERL at the CoRL workshop on Manipulation in a World of Abundant data at 9 am CET (Jupiter room)! Also leading discussion about cross-embodiment in the X-Embodiment workshop at 9:30 am (Terra) Come find out about cool recent advances!

Tmrw(Sat) I'll talk about pi-zero & HIL-SERL at the CoRL workshop on Manipulation in a World of Abundant data at 9 am CET (Jupiter room)! Also leading discussion about cross-embodiment in the X-Embodiment workshop at 9:30 am (Terra) Come find out about cool recent advances!

40,525 Aufrufe

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

We finished evaluating π0.7, our new model at Physical Intelligence. What I'm most excited about with π0.7 is that it's starting to show some surprising emergent compositional generalization, being able to both perform complex tasks and learn new tasks just from instructions.

We finished evaluating π0.7, our new model at Physical Intelligence. What I'm most excited about with π0.7 is that it's starting to show some surprising emergent compositional generalization, being able to both perform complex tasks and learn new tasks just from instructions.

59,885 Aufrufe • vor 1 Monat

A while back Benjie Holson described a set of "Robot Olympics" challenge tasks -- washing a pan, making a peanut butter sandwich, and more. We tried to fine-tune our models at PI to these tasks, and found that we could do most of them. A few highlights below.

A while back Benjie Holson described a set of "Robot Olympics" challenge tasks -- washing a pan, making a peanut butter sandwich, and more. We tried to fine-tune our models at PI to these tasks, and found that we could do most of them. A few highlights below.

81,204 Aufrufe • vor 5 Monaten

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

151,888 Aufrufe • vor 11 Monaten

One of my favorite results for π*0.6 is this video: 13 hours of making lattes, Americanos, and espresso for folks at our office in San Francisco.

One of my favorite results for π*0.6 is this video: 13 hours of making lattes, Americanos, and espresso for folks at our office in San Francisco.

57,855 Aufrufe • vor 6 Monaten

At Physical Intelligence, we teamed up with Weave Robotics and Ultra to stress-test our models in real-world deployments. This was a really fun collaboration that saw our latest pi06 model running in production at Sea Breeze Cleaners and a real warehouse! More below.

At Physical Intelligence, we teamed up with Weave Robotics and Ultra to stress-test our models in real-world deployments. This was a really fun collaboration that saw our latest pi06 model running in production at Sea Breeze Cleaners and a real warehouse! More below.

31,910 Aufrufe • vor 3 Monaten

Really excited to share what I've been working on with my colleagues at Physical Intelligence! We've developed a prototype robotic foundation model that can fold laundry, assemble a box, bus a table, and many other things. We've written a paper and blog post about it. 🧵👇

Really excited to share what I've been working on with my colleagues at Physical Intelligence! We've developed a prototype robotic foundation model that can fold laundry, assemble a box, bus a table, and many other things. We've written a paper and blog post about it. 🧵👇

114,931 Aufrufe • vor 1 Jahr

Real-time inference is a big challenge for VLAs. We’ve been working on a way to amortize inference delays in π0.5. Our new Real-Time Chunking (RTC) method speeds up π0.5 by allowing the robot to “think” while it’s moving, which makes it quite a bit faster! 🧵👇

Real-time inference is a big challenge for VLAs. We’ve been working on a way to amortize inference delays in π0.5. Our new Real-Time Chunking (RTC) method speeds up π0.5 by allowing the robot to “think” while it’s moving, which makes it quite a bit faster! 🧵👇

73,492 Aufrufe • vor 11 Monaten

Turns out that vision-language models can control robots too. The secret is to just finetune them to print out the actions (literally, as text). Really excited about our new result, the successor to RT-1. RT-2 is a pre-trained VLM: Short 🧵👇

Turns out that vision-language models can control robots too. The secret is to just finetune them to print out the actions (literally, as text). Really excited about our new result, the successor to RT-1. RT-2 is a pre-trained VLM: Short 🧵👇

165,052 Aufrufe • vor 2 Jahren

Benjie posted a reply to the Physical Intelligence "Olympics" attempt: Some nice discussion about what makes a task hard, how current learning methods should change how people think about robotic capability, and some interesting commentary.

Benjie posted a reply to the Physical Intelligence "Olympics" attempt: Some nice discussion about what makes a task hard, how current learning methods should change how people think about robotic capability, and some interesting commentary.

27,147 Aufrufe • vor 4 Monaten

Diffusion models make great images. But can they drive robots? Usually that gets complicated really fast. We figured out how to get a Stable Diffusion model (based on Instruct pix2pix) to drive robotic instruction following. Simple recipe, works on a wide range of tasks. Thread👇

Diffusion models make great images. But can they drive robots? Usually that gets complicated really fast. We figured out how to get a Stable Diffusion model (based on Instruct pix2pix) to drive robotic instruction following. Simple recipe, works on a wide range of tasks. Thread👇

126,519 Aufrufe • vor 2 Jahren

Very happy to announce that we are open-sourcing the π₀ model, weights, and some fine-tuned checkpoints! Hoping this leads to lots of great follow-up research: Here is a fun test from our friends at UPenn.

Very happy to announce that we are open-sourcing the π₀ model, weights, and some fine-tuned checkpoints! Hoping this leads to lots of great follow-up research: Here is a fun test from our friends at UPenn.

72,425 Aufrufe • vor 1 Jahr

If we train VLAs to respond to diverse multimodal prompts, then we can steer them better: [grasp the carrot]/[move to x,y,z]/[put the carrot on the plate]. With many levels of detail, powerful VLMs can step in and steer the model to success much more often! More below 👇

If we train VLAs to respond to diverse multimodal prompts, then we can steer them better: [grasp the carrot]/[move to x,y,z]/[put the carrot on the plate]. With many levels of detail, powerful VLMs can step in and steer the model to success much more often! More below 👇

20,933 Aufrufe • vor 3 Monaten

Language following is a tough problem for VLAs: while these models can follow complex language, in practice getting datasets that enable language following is hard. We developed a method to counterfactually and automatically label data to improve language following! 🧵👇

Language following is a tough problem for VLAs: while these models can follow complex language, in practice getting datasets that enable language following is hard. We developed a method to counterfactually and automatically label data to improve language following! 🧵👇

44,176 Aufrufe • vor 9 Monaten

Watch this robot dog learn to walk from scratch in real time! Our new method, APRL, dynamically adjusts exploration constraints to enable fast and performant RL directly in the real world. APRL can also adapt to changes in the terrain. No simulation, no demos. A thread 👇

Watch this robot dog learn to walk from scratch in real time! Our new method, APRL, dynamically adjusts exploration constraints to enable fast and performant RL directly in the real world. APRL can also adapt to changes in the terrain. No simulation, no demos. A thread 👇

105,568 Aufrufe • vor 2 Jahren

We trained a robotic foundation model that can drive mobile robots in six different countries, and navigate Sproul Plaza in midday on the UC Berkeley campus! Some cool new work w/ noriaki_hirose, Lydia Ignatova, Kyle Stachowicz, Catherine Glossop, Dhruv Shah

We trained a robotic foundation model that can drive mobile robots in six different countries, and navigate Sproul Plaza in midday on the UC Berkeley campus! Some cool new work w/ noriaki_hirose, Lydia Ignatova, Kyle Stachowicz, Catherine Glossop, Dhruv Shah

52,907 Aufrufe • vor 1 Jahr

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

48,667 Aufrufe • vor 11 Monaten

Scaling laws in deep RL? Turns out that batch size, learning rate, and UTD (update-to-data) for getting the most efficient and scalable deep RL has predictable relationships. Checkout the analysis in new work by Oleg Rybkin & collaborators:

Scaling laws in deep RL? Turns out that batch size, learning rate, and UTD (update-to-data) for getting the most efficient and scalable deep RL has predictable relationships. Checkout the analysis in new work by Oleg Rybkin & collaborators:

43,441 Aufrufe • vor 1 Jahr

RL in the real world presents some big challenges, but also some really big opportunities. In our new work, HIL-SERL, Charles Xu, Jeffrey Wu, Jianlan Luo show that real-world RL can learn a huge range of precise and robust tasks, and perform them much faster than imitation.

RL in the real world presents some big challenges, but also some really big opportunities. In our new work, HIL-SERL, Charles Xu, Jeffrey Wu, Jianlan Luo show that real-world RL can learn a huge range of precise and robust tasks, and perform them much faster than imitation.

36,489 Aufrufe • vor 1 Jahr

At Physical Intelligence, we have open roles robotics research scientist and research engineer. If you love robots, hands-on experimental robotics work, and getting amazing, dexterous skills to work, get in touch! See

At Physical Intelligence, we have open roles robotics research scientist and research engineer. If you love robots, hands-on experimental robotics work, and getting amazing, dexterous skills to work, get in touch! See

26,520 Aufrufe • vor 1 Jahr