Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We used interpretability to scale RL against open-ended tasks, cutting Gemma 12B’s hallucination rate in half by teaching it to self-correct in tandem with our probing harness.

Goodfire

26,367 subscribers

75,628 views • 6 months ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

439,920 views • 5 months ago

Want to scale RL with your shiny new GPU? 🚀 In our ICML24 Oral we find that RL algorithms hit a barrier when data is scaled up. Our new algorithm, SAPG, proposes a simple fix. It scales to 25k envs and solves hard tasks where PPO makes no progress. 1/n

Want to scale RL with your shiny new GPU? 🚀 In our ICML24 Oral we find that RL algorithms hit a barrier when data is scaled up. Our new algorithm, SAPG, proposes a simple fix. It scales to 25k envs and solves hard tasks where PPO makes no progress. 1/n

Ananye Agarwal

64,546 views • 2 years ago

1/ Introducing Lynx - the leading hallucination detection model 🚀👀 - Beats GPT-4o on hallucination tasks - Open source, open weights, open data - Excels in real-world domains like medicine and finance We are excited to launch Lynx with Day 1 integration partners: NVIDIA, MongoDB, and Nomic 🔥

1/ Introducing Lynx - the leading hallucination detection model 🚀👀 - Beats GPT-4o on hallucination tasks - Open source, open weights, open data - Excels in real-world domains like medicine and finance We are excited to launch Lynx with Day 1 integration partners: NVIDIA, MongoDB, and Nomic 🔥

PatronusAI

81,536 views • 2 years ago

Excited to announce RoboCasa, a large-scale simulation framework of everyday tasks! We use generative AI tools to create diverse objects, scenes, and tasks. Simulation plays a pivotal role in our Data Pyramid for training generalist robots. Open-source at

Excited to announce RoboCasa, a large-scale simulation framework of everyday tasks! We use generative AI tools to create diverse objects, scenes, and tasks. Simulation plays a pivotal role in our Data Pyramid for training generalist robots. Open-source at

Yuke Zhu

141,540 views • 2 years ago

Introducing /orchestrate, a skill that recursively spawns agents to tackle your most ambitious tasks with the Cursor SDK. We’ve used it to: - Autoresearch our internal skills, cutting token use by 20% while improving evals - Cut cold start times on our internal backend by 80%

Introducing /orchestrate, a skill that recursively spawns agents to tackle your most ambitious tasks with the Cursor SDK. We’ve used it to: - Autoresearch our internal skills, cutting token use by 20% while improving evals - Cut cold start times on our internal backend by 80%

Cursor

535,145 views • 3 months ago

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, Jagdeep Bhatia, Andrew Wagenmaker, Will Chen show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!

Sergey Levine

30,935 views • 1 month ago

Train your 🦞OpenClaw🦞 simply by talking to it. Meet OpenClaw-RL. Host your model on our RL server, and your LLM gets optimized automatically. Use it anywhere. Keep it private. Make it more personal every day. We have fully open sourced everything. Come in and have fun!

Train your 🦞OpenClaw🦞 simply by talking to it. Meet OpenClaw-RL. Host your model on our RL server, and your LLM gets optimized automatically. Use it anywhere. Keep it private. Make it more personal every day. We have fully open sourced everything. Come in and have fun!

Yinjie Wang

99,134 views • 5 months ago

GPT-5.6 Sol has been used to solve open problems in mathematics. So why was it struggling with ARC-AGI-3, a benchmark of 2D puzzle games? We investigated. The harness was not letting it remember what it had learned. We found that enabling two API settings tripled our scores with 6x fewer output tokens.

GPT-5.6 Sol has been used to solve open problems in mathematics. So why was it struggling with ARC-AGI-3, a benchmark of 2D puzzle games? We investigated. The harness was not letting it remember what it had learned. We found that enabling two API settings tripled our scores with 6x fewer output tokens.

OpenAI

1,216,953 views • 20 days ago

We must not allow our bases to be used by America in this illegal war against Iran. Instead of upholding International Law this Labour Government, in allowing our bases to be used, has become complicit and equally culpable!

We must not allow our bases to be used by America in this illegal war against Iran. Instead of upholding International Law this Labour Government, in allowing our bases to be used, has become complicit and equally culpable!

Ayoub Khan MP

13,347 views • 5 months ago

Introducing RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. 7 real robot tasks, 900/900 successes. Up to 250 consecutive trials in one task, running 2 hours nonstop without failure. High success rate against physical disturbances, zero-shot, and few-shot adaptation Our first step toward a deployable robot learning system.

Introducing RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. 7 real robot tasks, 900/900 successes. Up to 250 consecutive trials in one task, running 2 hours nonstop without failure. High success rate against physical disturbances, zero-shot, and few-shot adaptation Our first step toward a deployable robot learning system.

Kun Lei

91,496 views • 10 months ago

Jack Wilshere talking about how Arsene Wenger used unopposed tasks to train attacking combinations “He used to put loads of mannequins around the edge of the box, in groups of 3 we had to play around the mannequins with one touch” Self organizing patterns but with intent

Jack Wilshere talking about how Arsene Wenger used unopposed tasks to train attacking combinations “He used to put loads of mannequins around the edge of the box, in groups of 3 we had to play around the mannequins with one touch” Self organizing patterns but with intent

JOGO FUNCIONAL

1,213,215 views • 1 year ago

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨ We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict *when* hallucination happens and use that to fix it! 🧵1/n

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨ We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict when hallucination happens and use that to fix it! 🧵1/n

Nicklas Hansen

56,641 views • 1 month ago

Goal-conditioned RL (GCRL) is great - unsupervised, can use data (in offline mode), flexibility to define tasks at test time. But can we run GCRL on *language data*?? In our new work we show that language GCRL enables sophisticated test-time reasoning for interactive tasks! 🧵👇

Goal-conditioned RL (GCRL) is great - unsupervised, can use data (in offline mode), flexibility to define tasks at test time. But can we run GCRL on language data?? In our new work we show that language GCRL enables sophisticated test-time reasoning for interactive tasks! 🧵👇

Sergey Levine

18,782 views • 1 year ago

India is working on futuristic sectors to power growth and self-reliance. We are going to start a National Deepwater Exploration Mission to harness the potential in this sector. We will also be increasing our efforts in critical minerals.

India is working on futuristic sectors to power growth and self-reliance. We are going to start a National Deepwater Exploration Mission to harness the potential in this sector. We will also be increasing our efforts in critical minerals.

Narendra Modi

578,786 views • 1 year ago

🔗 You can now learn how to tie the Butterfly Harness step-by-step with our new Harness Guide: This popular arms-free pattern does it all - and yes, you can self-tie it! 📚 Versatile, visually striking, and a solid base for multiple positions, whether on the floor or in suspension. Once you’ve mastered it, the possibilities really open up, from bamboo and strappado variations to front, back, or side suspension.

Sensitive content

🔗 You can now learn how to tie the Butterfly Harness step-by-step with our new Harness Guide: This popular arms-free pattern does it all - and yes, you can self-tie it! 📚 Versatile, visually striking, and a solid base for multiple positions, whether on the floor or in suspension. Once you’ve mastered it, the possibilities really open up, from bamboo and strappado variations to front, back, or side suspension.

Shibari Study

90,183 views • 4 months ago

In our #ICML2023 paper (led by Ghada Sokar and with Rishabh Agarwal and utku ) we highlight the prevalence of dormant neurons in deep RL, and propose a simple method to mitigate it. Quick video summary here, and more details in thread!

In our #ICML2023 paper (led by Ghada Sokar and with Rishabh Agarwal and utku ) we highlight the prevalence of dormant neurons in deep RL, and propose a simple method to mitigate it. Quick video summary here, and more details in thread!

Pablo Samuel Castro

40,603 views • 3 years ago

I've long wondered if we can make a humanoid robot do a 𝘄𝗮𝗹𝗹𝗳𝗹𝗶𝗽 - and we just made it happen by leveraging 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁 with BeyondMimic tracking! This came after our original OmniRetarget experiments, with only minor tweaks to RL training: relaxing a termination threshold and removing one reward term. The policy achieved a 𝟱/𝟱 success rate in our real-world experiments, showing the strength of high-quality, interaction-preserving motion retargeting combined with BeyondMimic’s minimal RL tracking. Here is the updated arXiv: (In Sec. V. A)

I've long wondered if we can make a humanoid robot do a 𝘄𝗮𝗹𝗹𝗳𝗹𝗶𝗽 - and we just made it happen by leveraging 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁 with BeyondMimic tracking! This came after our original OmniRetarget experiments, with only minor tweaks to RL training: relaxing a termination threshold and removing one reward term. The policy achieved a 𝟱/𝟱 success rate in our real-world experiments, showing the strength of high-quality, interaction-preserving motion retargeting combined with BeyondMimic’s minimal RL tracking. Here is the updated arXiv: (In Sec. V. A)

Zhen Wu

1,051,448 views • 10 months ago

We finally got our visas. Now it’s time to open our foundry in the USA! We are looking for investors to walk by our side in this adventure!

We finally got our visas. Now it’s time to open our foundry in the USA! We are looking for investors to walk by our side in this adventure!

Atelier Missor

654,867 views • 2 months ago