Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

svlevine's profile picture

Sergey Levine

138,125 subscribers

48,840 görüntüleme • 1 yıl önce •via X (Twitter)

Bilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

5 Yorum

Sergey Levine profil fotoğrafı
Sergey Levine1 yıl önce

Imagine an MDP where the state is the current crop of the image, an action is to pick a new crop, and rewards are matching textual captions or other (weak or strong) labels. Training a value function for this MDP instantiations a representation learning method.

Sergey Levine profil fotoğrafı
Sergey Levine1 yıl önce

Reward could come from matching a text label, or provided in a fully unsupervised way via crop consistency. The stronger the reward, the better it works, but even weak rewards like crop consistency lead to improvement. For more, check out the website:

Joanne Mercado profil fotoğrafı
Joanne Mercado1 yıl önce

@its_dibya *an SSL, but overall your grammar and punctuation are top-tier 💯

Ethan vs Machines profil fotoğrafı
Ethan vs Machines1 yıl önce

@its_dibya RL for SSL using semantic rewards? Brilliant method. Scaling beyond COCO might be tough here though—Canada’s R&D can’t keep up with compute demands anymore.

ᐸGerardSans/ᐳ🚀🇬🇧 profil fotoğrafı
ᐸGerardSans/ᐳ🚀🇬🇧1 yıl önce

@its_dibya That’s just flattened patching which is something but not really.

Benzer Videolar

Our work, "A Primer on SO(3) Action Representations in Deep Reinforcement Learning," was accepted to #ICLR2026! We provide a systematic study of action representation choices in RL, showing that they fundamentally impact training stability and performance. #Robotics #AI #RL
0:21

Our work, "A Primer on SO(3) Action Representations in Deep Reinforcement Learning," was accepted to #ICLR2026! We provide a systematic study of action representation choices in RL, showing that they fundamentally impact training stability and performance. #Robotics #AI #RL

Learning Systems and Robotics Lab (is hiring!)

50,130 görüntüleme • 8 ay önce

✨ICLR 2024 Spotlight Still hand-labeling phases or tuning GANs with your trajectories? We propose FLD, a self-supervised representation method that extracts spatial-temporal relationships in high-dimensional trajectories. RL policies informed by FLD show extended generality.🧵
1:57

✨ICLR 2024 Spotlight Still hand-labeling phases or tuning GANs with your trajectories? We propose FLD, a self-supervised representation method that extracts spatial-temporal relationships in high-dimensional trajectories. RL policies informed by FLD show extended generality.🧵

Chenhao Li

81,988 görüntüleme • 2 yıl önce

NeurIPS 2025 Paper: LLMs are Reinforcement Learners 🤯! Surprisingly, we show that LLMs can solve RL tasks without any external component! We introduce Prompted Policy Search (ProPS), an RL method based only LLMs and in-context learning. [Paper]
0:44

NeurIPS 2025 Paper: LLMs are Reinforcement Learners 🤯! Surprisingly, we show that LLMs can solve RL tasks without any external component! We introduce Prompted Policy Search (ProPS), an RL method based only LLMs and in-context learning. [Paper]

Heni Ben Amor

51,248 görüntüleme • 11 ay önce

Scaling laws in deep RL? Turns out that batch size, learning rate, and UTD (update-to-data) for getting the most efficient and scalable deep RL has predictable relationships. Checkout the analysis in new work by Oleg Rybkin & collaborators:
0:29

Scaling laws in deep RL? Turns out that batch size, learning rate, and UTD (update-to-data) for getting the most efficient and scalable deep RL has predictable relationships. Checkout the analysis in new work by Oleg Rybkin & collaborators:

Sergey Levine

43,494 görüntüleme • 1 yıl önce

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning *beyond* just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)
0:48

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning *beyond* just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

Abhishek Gupta

13,637 görüntüleme • 1 yıl önce

🚨 New (Offline) RL Method 🦾 🚨 Introducing Q-Transformer - new RL approach that works at scale with large models and many tasks. This is the best method we found so far that works with demos and autonomous (also negative) data at large scale. 🧵
0:22

🚨 New (Offline) RL Method 🦾 🚨 Introducing Q-Transformer - new RL approach that works at scale with large models and many tasks. This is the best method we found so far that works with demos and autonomous (also negative) data at large scale. 🧵

Karol Hausman

56,054 görüntüleme • 3 yıl önce

What do we really need for social navigation?🤖 Deep RL? No❌ Imitation learning? No❌ LLMs? No❌ 📊Turns out for over 80% of the time, classical methods >> learning-based solutions! In new work, we rethink social navigation. Thread🧶
1:58

What do we really need for social navigation?🤖 Deep RL? No❌ Imitation learning? No❌ LLMs? No❌ 📊Turns out for over 80% of the time, classical methods >> learning-based solutions! In new work, we rethink social navigation. Thread🧶

Rohan Chandra

14,067 görüntüleme • 3 yıl önce

We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & Andrew Wagenmaker
0:46

We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & Andrew Wagenmaker

Sergey Levine

38,514 görüntüleme • 4 ay önce

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇
0:39

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

Sergey Levine

153,232 görüntüleme • 1 yıl önce

Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another? We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵
1:02

Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another? We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵

Wilka Carvalho

15,333 görüntüleme • 4 ay önce

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵
0:39

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

Gabriel Sarch

76,824 görüntüleme • 1 yıl önce

Every wondered if we can model motion as a language? can we tokenize this new language? is it useful? Turns out tremendously! 🚀 In out latest #NeurIPS2024 paper on QueST: Self-Supervised Skill Abstractions for Learning Continuous Control, we find that action tokenization matters a lot! We can learn skill encodings by representing temporal action abstractions with a discrete codebook. This enables 2 things 1. Better Behaviour Cloning: we can better assimilate multi-task data (>9%) over best paper. This is currently best in class BC method! 2. generalization of this language to represent new tasks in 5-shot transfer to longer horizon tasks! Check out the thread by Atharva Mete for more details. And check out more details at: Joint work with Atharva Mete Albert Wilcox Haotian Xue Yongxin Chen Georgia Tech School of Interactive Computing Machine Learning at Georgia Tech Robotics@GT NVIDIA Robotics
1:10

Every wondered if we can model motion as a language? can we tokenize this new language? is it useful? Turns out tremendously! 🚀 In out latest #NeurIPS2024 paper on QueST: Self-Supervised Skill Abstractions for Learning Continuous Control, we find that action tokenization matters a lot! We can learn skill encodings by representing temporal action abstractions with a discrete codebook. This enables 2 things 1. Better Behaviour Cloning: we can better assimilate multi-task data (>9%) over best paper. This is currently best in class BC method! 2. generalization of this language to represent new tasks in 5-shot transfer to longer horizon tasks! Check out the thread by Atharva Mete for more details. And check out more details at: Joint work with Atharva Mete Albert Wilcox Haotian Xue Yongxin Chen Georgia Tech School of Interactive Computing Machine Learning at Georgia Tech Robotics@GT NVIDIA Robotics

Animesh Garg

26,254 görüntüleme • 2 yıl önce

Eric Jang weighs in on RL vs. supervised learning for humanoid robot manipulation tasks.
1:40

Eric Jang weighs in on RL vs. supervised learning for humanoid robot manipulation tasks.

The Humanoid Hub

11,835 görüntüleme • 1 yıl önce

How do we run RL with real-time chunking (RTC)? In this work we figured out how to use a small RL policy with a large robot foundation model, where the RL policy observed more recent images (due to faster inference) and steers the policy toward better behaviors! A fun collaboration with Siemens, led by Brian Zhu, Momen Khalil, Emanuele Poggi from Siemens and E Harrison from Berkeley, with lots of amazing contributors!
3:21

How do we run RL with real-time chunking (RTC)? In this work we figured out how to use a small RL policy with a large robot foundation model, where the RL policy observed more recent images (due to faster inference) and steers the policy toward better behaviors! A fun collaboration with Siemens, led by Brian Zhu, Momen Khalil, Emanuele Poggi from Siemens and E Harrison from Berkeley, with lots of amazing contributors!

Sergey Levine

46,467 görüntüleme • 1 ay önce

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
2:15

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

447,445 görüntüleme • 7 ay önce

I am excited to present you with a Christmas gift :P 😆😆 Our new fine-tunable generalist agent is what we call the Pangu-Agent! Our agent is fantastic - ok! It supports the following: 1. Structured reasoning via a new notion of intrinsic functions that allows you to change the internal structure of your agent to better suit your task - in fact, we think this notion is critical to scale reinforcement learning to generalisable setups. Those functions are easy to write, compose, fine-tune, nest, and do whatever you like to do with them. 2. With those, we have a framework that allows you to consider prompt engineering methods as a particular case nesting of intrinsic forms - in other words, those prompting techniques we have, ReACT, ZS-CoT and so forth, are just a particular case. Now, we can make new ones with simple lines of code. 3. All agents need to learn new skills and adapt. Agents without abilities to fine-tune are not agents. They are as good as your pre-trained LLM. Well, we now can fine-tune using three different techniques: 3.1) Standard supervised fine-tuning; 3.2) [New] Bootstrapped supervised fine-tuning (mix-n-match) intrinsic functions; and 3.3) Reinforcement learning fine-tuning [We did so many tricks here to get RL fine-tuning to work well] 4. We show awesome zero-shot transfer results without ingesting #GPT4 data. Just using open-source models! 5. We also show that you can train one cross-domain model across BabyAI and AlfWorld tasks, fine-tune with RL, and perform well in held-out tasks. 6. What is also nice is that we can now have rigorous benchmarking experimentation as well - it seems not all that is reported is what really works :P Paper: #AI #MachineLearning This work has been done by a fantastic team, which I am proud to co-lead and work with! This project has been managed by the amazing Kun Shao , who has been impressive, to say the least! Technically supervised by Professor Jun Wang, who had the view of where we needed to go and guided by Professor Hao from tech and application aspects!! Wow! Nice end of the year!!!
3:40

I am excited to present you with a Christmas gift :P 😆😆 Our new fine-tunable generalist agent is what we call the Pangu-Agent! Our agent is fantastic - ok! It supports the following: 1. Structured reasoning via a new notion of intrinsic functions that allows you to change the internal structure of your agent to better suit your task - in fact, we think this notion is critical to scale reinforcement learning to generalisable setups. Those functions are easy to write, compose, fine-tune, nest, and do whatever you like to do with them. 2. With those, we have a framework that allows you to consider prompt engineering methods as a particular case nesting of intrinsic forms - in other words, those prompting techniques we have, ReACT, ZS-CoT and so forth, are just a particular case. Now, we can make new ones with simple lines of code. 3. All agents need to learn new skills and adapt. Agents without abilities to fine-tune are not agents. They are as good as your pre-trained LLM. Well, we now can fine-tune using three different techniques: 3.1) Standard supervised fine-tuning; 3.2) [New] Bootstrapped supervised fine-tuning (mix-n-match) intrinsic functions; and 3.3) Reinforcement learning fine-tuning [We did so many tricks here to get RL fine-tuning to work well] 4. We show awesome zero-shot transfer results without ingesting #GPT4 data. Just using open-source models! 5. We also show that you can train one cross-domain model across BabyAI and AlfWorld tasks, fine-tune with RL, and perform well in held-out tasks. 6. What is also nice is that we can now have rigorous benchmarking experimentation as well - it seems not all that is reported is what really works :P Paper: #AI #MachineLearning This work has been done by a fantastic team, which I am proud to co-lead and work with! This project has been managed by the amazing Kun Shao , who has been impressive, to say the least! Technically supervised by Professor Jun Wang, who had the view of where we needed to go and guided by Professor Hao from tech and application aspects!! Wow! Nice end of the year!!!

Haitham Bou Ammar

15,534 görüntüleme • 2 yıl önce

She looks like a rl barbie in this movie imo
0:20

She looks like a rl barbie in this movie imo

scarlettxmee

19,401 görüntüleme • 10 ay önce

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!
1:22

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

Sergey Levine

32,162 görüntüleme • 3 ay önce

Goal-conditioned RL (GCRL) is great - unsupervised, can use data (in offline mode), flexibility to define tasks at test time. But can we run GCRL on *language data*?? In our new work we show that language GCRL enables sophisticated test-time reasoning for interactive tasks! 🧵👇
0:22

Goal-conditioned RL (GCRL) is great - unsupervised, can use data (in offline mode), flexibility to define tasks at test time. But can we run GCRL on *language data*?? In our new work we show that language GCRL enables sophisticated test-time reasoning for interactive tasks! 🧵👇

Sergey Levine

18,782 görüntüleme • 1 yıl önce