Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

Aayush Karan

1,675 subscribers

278,167 views • 10 months ago •via X (Twitter)

Health & Wellness Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

No better way to kick off the weekend than with Lady Gaga’s new concert special. (Re)watch MAYHEM Requiem:

No better way to kick off the weekend than with Lady Gaga’s new concert special. (Re)watch MAYHEM Requiem:

Gaga Daily

19,627 views • 3 months ago

No better way to wrap up our America 250 celebration than with a sick drone show 🔥

No better way to wrap up our America 250 celebration than with a sick drone show 🔥

Baltimore Orioles

35,161 views • 1 month ago

I think I found a way to make a React Native list component with significantly better performance than FlashList, with 100% javascript. I'm still testing it but as far as I can tell there’s no pitfalls or issues… It just requires the new architecture for synchronous onLayout.

I think I found a way to make a React Native list component with significantly better performance than FlashList, with 100% javascript. I'm still testing it but as far as I can tell there’s no pitfalls or issues… It just requires the new architecture for synchronous onLayout.

Jay Meistrich

95,385 views • 1 year ago

New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.

New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.

Databricks AI Research

12,699 views • 5 months ago

This week we did a little low crawl training while getting shot at by airsoft. Is it the best training in the history of forever? No. Do you want to get shot at point plank with an airsoft bb cause you weren’t staying low enough to the ground? Also no.

This week we did a little low crawl training while getting shot at by airsoft. Is it the best training in the history of forever? No. Do you want to get shot at point plank with an airsoft bb cause you weren’t staying low enough to the ground? Also no.

PNWGUERRILLA

84,070 views • 1 month ago

No video on Earth lays out the Donald Trump political base better than this one. And if you think these folks can be talked to, reasoned with or extracted from the cult you're also delusional.

No video on Earth lays out the Donald Trump political base better than this one. And if you think these folks can be talked to, reasoned with or extracted from the cult you're also delusional.

Don Winslow

3,996,947 views • 3 years ago

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and self-divergence. In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.

Minqi Jiang

41,099 views • 11 months ago

No better way to set a career high in 3s than a ridiculous assist from Sengun. 7 3s for Reed a new career high.

No better way to set a career high in 3s than a ridiculous assist from Sengun. 7 3s for Reed a new career high.

Lachard Binkley

11,972 views • 5 months ago

HAPPY COIN DAY, VIKINGS! 🥳 There is no better way to celebrate than with a JOKER GAME! 🃏✨ Can you tell where it is hiding? Is it A, B, or C? Tell us in the comments! 👇

HAPPY COIN DAY, VIKINGS! 🥳 There is no better way to celebrate than with a JOKER GAME! 🃏✨ Can you tell where it is hiding? Is it A, B, or C? Tell us in the comments! 👇

Coin Master

34,570 views • 6 months ago

🚀 Introducing Boosts – a new way for Cardano projects to gain traction and visibility on TapTools. We built this to give founders a better shot at exposure—more eyes, more engagement, and more growth. No gatekeeping. No barriers. Just visibility where it matters.

🚀 Introducing Boosts – a new way for Cardano projects to gain traction and visibility on TapTools. We built this to give founders a better shot at exposure—more eyes, more engagement, and more growth. No gatekeeping. No barriers. Just visibility where it matters.

TapTools

19,632 views • 1 year ago

Julius Malema is angry that we have better organisers than his EFF cult. Our people have demonstrated how better are we when there’s no KFC or buses involved. This is 100% authentic, no one is going to lose a job for not bringing people from their community with a bus.

Julius Malema is angry that we have better organisers than his EFF cult. Our people have demonstrated how better are we when there’s no KFC or buses involved. This is 100% authentic, no one is going to lose a job for not bringing people from their community with a bus.

Jeff Ngobeni

28,282 views • 2 months ago

I can hit 200m, standing, with my offset holoson 507C redot on my AR-15. In no way, shape, or form is 200m an impressive shot to make with a scoped bolt action rifle. Doesn’t take “a skilled sniper”. (Second video for better understanding of where my offset red dot is on my rifle)

I can hit 200m, standing, with my offset holoson 507C redot on my AR-15. In no way, shape, or form is 200m an impressive shot to make with a scoped bolt action rifle. Doesn’t take “a skilled sniper”. (Second video for better understanding of where my offset red dot is on my rifle)

PNWGUERRILLA

336,877 views • 11 months ago

No better way to celebrate signing a new deal then to take over bath time for Mr. Brody (with a little singing of course) 😂 #WhiteRhino #TrenchMafia

No better way to celebrate signing a new deal then to take over bath time for Mr. Brody (with a little singing of course) 😂 #WhiteRhino #TrenchMafia

Nikki Bozeman

73,857 views • 3 years ago

No one can give a better reply to Dhruv Rathi than Ajit Bharti (with Facts and Sarcasm) Not even the BJP IT cell. #Dhruv_Rathee

No one can give a better reply to Dhruv Rathi than Ajit Bharti (with Facts and Sarcasm) Not even the BJP IT cell. #Dhruv_Rathee

Saffronist Ghost

259,197 views • 2 years ago

There is no better day than today. There is no better time than now. Welcome to the future of finance with WLFI's USD1 now on the high-speed rails of Aptos, here to deliver a stable, transparent, and accessible foundation for the digital economy to all. Accelerate 🦅

There is no better day than today. There is no better time than now. Welcome to the future of finance with WLFI's USD1 now on the high-speed rails of Aptos, here to deliver a stable, transparent, and accessible foundation for the digital economy to all. Accelerate 🦅

Aptos

910,039 views • 10 months ago

There is no better team to watch in regard to playing everything from a 2-High shell than the Panthers. No one does it more often than Carolina. The trend in the NFL is to marry Weak Rotation Cover 3 with Quarters or a Fangio-style "rotational" split field system. --

There is no better team to watch in regard to playing everything from a 2-High shell than the Panthers. No one does it more often than Carolina. The trend in the NFL is to marry Weak Rotation Cover 3 with Quarters or a Fangio-style "rotational" split field system. --

Cody Alexander

30,956 views • 4 months ago

Always innovating morning, noon and night. 🌅☀️🌙 There’s no better way to celebrate Dell’s 40th anniversary than with a kaleidoscope on the coolest LED screen in the world. ✨ #DellTechWorld Sphere

Always innovating morning, noon and night. 🌅☀️🌙 There’s no better way to celebrate Dell’s 40th anniversary than with a kaleidoscope on the coolest LED screen in the world. ✨ #DellTechWorld Sphere

Dell Technologies

12,489 views • 2 years ago

BIG NEWS: Hasbro is joining Team Massachusetts! For an iconic brand that brings joy to children everywhere, there's no better home base than the best state for innovation, education and families. And with hundreds of new jobs on the way, this is a huge win for our state.

Governor Maura Healey

23,280 views • 11 months ago