Bye bye looped reasoning… and hello Energy-Based Reasoning⚡!
@AlexiGlad10K followers
163,145 views
27 Comments
Comments from the original post on X (Twitter)

@AlexiGlad
[1/N] So what’s missing in looped reasoning models? 🧐 HRM and TRM loop a small block to refine their answer, so they think longer without getting bigger But nothing checks how good the current answer is Stopping and choosing an answer needs add-ons like a halting head, or many samples + voting So we ask: can a learned objective guide every update, decide when to stop, and verify which answer is best?
21 likes

@AlexiGlad
[2/N]💡EBRs solve this by reasoning via verification A learned energy scores each answer, and gradient descent on that energy updates it. Each step rolls downhill to a better answer When the score stops getting better, we stop. Then we pick the states with the lowest energy
15 likes

@AlexiGlad
[3/N] What can I use this energy for? The same energy tells EBRs when to stop and which answers are best 🛑 Stop: once energy is flat. Easy puzzles finish early, so more steps can be used for harder puzzles 🎯 Pick: incorrect answers have higher energy, so among several samples we keep the lowest-energy one This allows us to achieve a 345x speedup over the next best model
11 likes

@AlexiGlad
[4/N] 📈How does EBR compare and scale with more compute? EBR gets 99.9% on Sudoku and 88.3% on Maze-hard EBRs also scale much better than other reasoning models at inference time: 🔁 7× fewer iterations 🎲 43× fewer samples (3 vs 128)
10 likes

@AlexiGlad
[5/N] 🎲 Many reasoning models combine generative modeling to reasoning But this often adds exposure bias, which reduces generalization… To solve this, we extend Explorative Modeling (XM) to train latent reasoning models, which allows for modeling uncertainty and diversity at inference time See XM:
11 likes

@AlexiGlad
Check out the rest of this thread from the super smart @wwillchen1 who led this work (and is also applying for PhDs this cycle)!
10 likes

@DanielLenton1
Could you add a new constraint at inference just by adding another energy term?
4 likes

@AlexiGlad
yes! one of the core benefits is you can do compositional generalization with EBMs (example here we didn't test that here but would definitely be sick to try for reasoning
5 likes

@Jacklu_me
solving sudokus cheaper than llms can even verify them
4 likes

@AlexiGlad
💨

@TrelisResearch
What’s largest model trained with EBR?
2 likes

@AlexiGlad
actually, we trained very small models < 10M params! we were focused on efficient reasoning and generalization, as these benchmarks focus on that, but scaling up for language models is coming we will also release arc agi results soon :)
2 likes

@huskydogewoof
Congratulations! I heard about this from Yilun at ICML; it is super cool! I personally tried to play with EBM when I was working on EqR, but I had some trouble finding the right hyperparameters to go with (this should be a skill issue). Also heard from @wwillchen1 that you are working on ARC-AGI. Looking forward to it!
2 likes

@CharlesRDog
@beffjezos
1 like

@MrCatid
Compared to EBR, CRAM, basic counting, and Weight-Space Programming are significantly more accurate and faster to train on standard reasoning benchmarks like Sudoku-Extreme.
1 like

@AlexiGlad
hey! agreed regarding the results, but imo, this is goodhart's law coming into effect, and using non ANN approaches defeats the purpose of these benchmarks sudoku solvers will always be optimal here (we know how to solve sudoku!), so i think testing approaches that won't scale as well (non deep ANN approaches) is somewhat meaningless given that you need to learn everything with a deep ANN, i believe EBR is SOTA!
2 likes

@MrCatid
Good points I’d like to see people attack harder problems these simple approaches start being worse at
1 like

@thefusioncube
But does it scale…?
1 like

@AlexiGlad
we are actively cooking here🧑🍳!
1 like

@markshi07
curious how this transfers to tasks without a clean right answer. does the energy stay meaningful for open-ended generation?





