Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

We found a new way to get language models to reason. 🤯 No RL, no training, no verifiers, no prompting. ❌ With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

aakaran31's profile picture

Aayush Karan

2,899 subscribers

280,576 görüntüleme • 11 ay önce •via X (Twitter)

Sağlık & İyilikBilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog:
0:10

Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog:

Sophie Wang

103,738 görüntüleme • 1 gün önce

Flow Reasoning Models are looped models. In our recent work, we turn a flow model into a recurrent model to solve reasoning problems (e.g. Sudoku). Predictions are fed back as inputs and iteratively refined. Each pass gets its own local loss. No backprop through time.
0:19

Flow Reasoning Models are looped models. In our recent work, we turn a flow model into a recurrent model to solve reasoning problems (e.g. Sudoku). Predictions are fed back as inputs and iteratively refined. Each pass gets its own local loss. No backprop through time.

Alec Helbling

33,800 görüntüleme • 20 gün önce

No better way to kick off the weekend than with Lady Gaga’s new concert special. (Re)watch MAYHEM Requiem:
0:16

No better way to kick off the weekend than with Lady Gaga’s new concert special. (Re)watch MAYHEM Requiem:

Gaga Daily

20,789 görüntüleme • 4 ay önce

No better way to wrap up our America 250 celebration than with a sick drone show 🔥
0:14

No better way to wrap up our America 250 celebration than with a sick drone show 🔥

Baltimore Orioles

35,161 görüntüleme • 2 ay önce

I think I found a way to make a React Native list component with significantly better performance than FlashList, with 100% javascript. I'm still testing it but as far as I can tell there’s no pitfalls or issues… It just requires the new architecture for synchronous onLayout.
0:13

I think I found a way to make a React Native list component with significantly better performance than FlashList, with 100% javascript. I'm still testing it but as far as I can tell there’s no pitfalls or issues… It just requires the new architecture for synchronous onLayout.

Jay Meistrich

95,414 görüntüleme • 1 yıl önce

New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.
0:13

New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.

Databricks AI Research

12,823 görüntüleme • 7 ay önce

This week we did a little low crawl training while getting shot at by airsoft. Is it the best training in the history of forever? No. Do you want to get shot at point plank with an airsoft bb cause you weren’t staying low enough to the ground? Also no.
0:19

This week we did a little low crawl training while getting shot at by airsoft. Is it the best training in the history of forever? No. Do you want to get shot at point plank with an airsoft bb cause you weren’t staying low enough to the ground? Also no.

PNWGUERRILLA

84,070 görüntüleme • 2 ay önce

No video on Earth lays out the Donald Trump political base better than this one. And if you think these folks can be talked to, reasoned with or extracted from the cult you're also delusional.
0:10

No video on Earth lays out the Donald Trump political base better than this one. And if you think these folks can be talked to, reasoned with or extracted from the cult you're also delusional.

Don Winslow

3,997,328 görüntüleme • 3 yıl önce

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.
0:10

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.

Minqi Jiang

41,154 görüntüleme • 1 yıl önce

No better way to set a career high in 3s than a ridiculous assist from Sengun. 7 3s for Reed a new career high.
0:15

No better way to set a career high in 3s than a ridiculous assist from Sengun. 7 3s for Reed a new career high.

Lachard Binkley

11,972 görüntüleme • 7 ay önce

HAPPY COIN DAY, VIKINGS! 🥳 There is no better way to celebrate than with a JOKER GAME! 🃏✨ Can you tell where it is hiding? Is it A, B, or C? Tell us in the comments! 👇
0:10

HAPPY COIN DAY, VIKINGS! 🥳 There is no better way to celebrate than with a JOKER GAME! 🃏✨ Can you tell where it is hiding? Is it A, B, or C? Tell us in the comments! 👇

Coin Master

35,106 görüntüleme • 8 ay önce

🚀 Introducing Boosts – a new way for Cardano projects to gain traction and visibility on TapTools. We built this to give founders a better shot at exposure—more eyes, more engagement, and more growth. No gatekeeping. No barriers. Just visibility where it matters.
0:14

🚀 Introducing Boosts – a new way for Cardano projects to gain traction and visibility on TapTools. We built this to give founders a better shot at exposure—more eyes, more engagement, and more growth. No gatekeeping. No barriers. Just visibility where it matters.

TapTools

19,636 görüntüleme • 1 yıl önce

Julius Malema is angry that we have better organisers than his EFF cult. Our people have demonstrated how better are we when there’s no KFC or buses involved. This is 100% authentic, no one is going to lose a job for not bringing people from their community with a bus.
0:12

Julius Malema is angry that we have better organisers than his EFF cult. Our people have demonstrated how better are we when there’s no KFC or buses involved. This is 100% authentic, no one is going to lose a job for not bringing people from their community with a bus.

Jeff Ngobeni

28,282 görüntüleme • 4 ay önce

I can hit 200m, standing, with my offset holoson 507C redot on my AR-15. In no way, shape, or form is 200m an impressive shot to make with a scoped bolt action rifle. Doesn’t take “a skilled sniper”. (Second video for better understanding of where my offset red dot is on my rifle)
0:09

I can hit 200m, standing, with my offset holoson 507C redot on my AR-15. In no way, shape, or form is 200m an impressive shot to make with a scoped bolt action rifle. Doesn’t take “a skilled sniper”. (Second video for better understanding of where my offset red dot is on my rifle)

PNWGUERRILLA

337,014 görüntüleme • 1 yıl önce

No better way to celebrate signing a new deal then to take over bath time for Mr. Brody (with a little singing of course) 😂 #WhiteRhino #TrenchMafia
0:19

No better way to celebrate signing a new deal then to take over bath time for Mr. Brody (with a little singing of course) 😂 #WhiteRhino #TrenchMafia

Nikki Bozeman

73,857 görüntüleme • 3 yıl önce

No one can give a better reply to Dhruv Rathi than Ajit Bharti (with Facts and Sarcasm) Not even the BJP IT cell. #Dhruv_Rathee
0:13

No one can give a better reply to Dhruv Rathi than Ajit Bharti (with Facts and Sarcasm) Not even the BJP IT cell. #Dhruv_Rathee

Saffronist Ghost

259,197 görüntüleme • 2 yıl önce

There is no better day than today. There is no better time than now. Welcome to the future of finance with WLFI's USD1 now on the high-speed rails of Aptos, here to deliver a stable, transparent, and accessible foundation for the digital economy to all. Accelerate 🦅
0:17

There is no better day than today. There is no better time than now. Welcome to the future of finance with WLFI's USD1 now on the high-speed rails of Aptos, here to deliver a stable, transparent, and accessible foundation for the digital economy to all. Accelerate 🦅

Aptos

910,197 görüntüleme • 1 yıl önce

There is no better team to watch in regard to playing everything from a 2-High shell than the Panthers. No one does it more often than Carolina. The trend in the NFL is to marry Weak Rotation Cover 3 with Quarters or a Fangio-style "rotational" split field system. --
0:16

There is no better team to watch in regard to playing everything from a 2-High shell than the Panthers. No one does it more often than Carolina. The trend in the NFL is to marry Weak Rotation Cover 3 with Quarters or a Fangio-style "rotational" split field system. --

Cody Alexander

30,956 görüntüleme • 5 ay önce

Always innovating morning, noon and night. 🌅☀️🌙 There’s no better way to celebrate Dell’s 40th anniversary than with a kaleidoscope on the coolest LED screen in the world. ✨ #DellTechWorld Sphere
0:19

Always innovating morning, noon and night. 🌅☀️🌙 There’s no better way to celebrate Dell’s 40th anniversary than with a kaleidoscope on the coolest LED screen in the world. ✨ #DellTechWorld Sphere

Dell Technologies

12,489 görüntüleme • 2 yıl önce