Skip navigation
Bye bye looped reasoning… and hello Energy-Based Reasoning⚡!
0:00 / 0:52

Bye bye looped reasoning… and hello Energy-Based Reasoning⚡!

Alexi Gladstone

@AlexiGlad10K followers

Visit on X
2,767 likesDownloadOriginal post

163,145 views

Bye bye looped reasoning… and hello Energy-Based Reasoning⚡! Introducing Energy-Based Reasoners TLDR: - EBRs use 345x less inference compute and 2.6x fewer training FLOPs than the best baselines - EBRs are 7x-43x more sample-efficient than the best looped baselines at inference - EBRs achieve 99.9% on Sudoku-Extreme and near 100% accuracy on multi-solution benchmarks 🧵Thread

27 Comments

Comments from the original post on X (Twitter)

  • Alexi Gladstone's profile picture

    @AlexiGlad

    [1/N] So what’s missing in looped reasoning models? 🧐 HRM and TRM loop a small block to refine their answer, so they think longer without getting bigger But nothing checks how good the current answer is Stopping and choosing an answer needs add-ons like a halting head, or many samples + voting So we ask: can a learned objective guide every update, decide when to stop, and verify which answer is best?

    21 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    [2/N]💡EBRs solve this by reasoning via verification A learned energy scores each answer, and gradient descent on that energy updates it. Each step rolls downhill to a better answer When the score stops getting better, we stop. Then we pick the states with the lowest energy

    15 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    [3/N] What can I use this energy for? The same energy tells EBRs when to stop and which answers are best 🛑 Stop: once energy is flat. Easy puzzles finish early, so more steps can be used for harder puzzles 🎯 Pick: incorrect answers have higher energy, so among several samples we keep the lowest-energy one This allows us to achieve a 345x speedup over the next best model

    11 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    [4/N] 📈How does EBR compare and scale with more compute? EBR gets 99.9% on Sudoku and 88.3% on Maze-hard EBRs also scale much better than other reasoning models at inference time: 🔁 7× fewer iterations 🎲 43× fewer samples (3 vs 128)

    10 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    [5/N] 🎲 Many reasoning models combine generative modeling to reasoning But this often adds exposure bias, which reduces generalization… To solve this, we extend Explorative Modeling (XM) to train latent reasoning models, which allows for modeling uncertainty and diversity at inference time See XM:

    11 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    Check out the rest of this thread from the super smart @wwillchen1 who led this work (and is also applying for PhDs this cycle)!

    10 likes

  • Daniel Lenton's profile picture

    @DanielLenton1

    Could you add a new constraint at inference just by adding another energy term?

    4 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    yes! one of the core benefits is you can do compositional generalization with EBMs (example here we didn't test that here but would definitely be sick to try for reasoning

    5 likes

  • Jack Lu ✈️ NeurIPS 26's profile picture

    @Jacklu_me

    solving sudokus cheaper than llms can even verify them

    4 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    💨

  • Trelis Research's profile picture

    @TrelisResearch

    What’s largest model trained with EBR?

    2 likes

  • Alexi Gladstone's profile picture

    @AlexiGlad

    actually, we trained very small models < 10M params! we were focused on efficient reasoning and generalization, as these benchmarks focus on that, but scaling up for language models is coming we will also release arc agi results soon :)

    2 likes

  • Benhao Huang's profile picture

    @huskydogewoof

    Congratulations! I heard about this from Yilun at ICML; it is super cool! I personally tried to play with EBM when I was working on EqR, but I had some trouble finding the right hyperparameters to go with (this should be a skill issue). Also heard from @wwillchen1 that you are working on ARC-AGI. Looking forward to it!

    2 likes

  • Mr Clanker McSlop Cannon the 3rd's profile picture

    @CharlesRDog

    @beffjezos

    1 like

  • catid's profile picture

    @MrCatid

    Compared to EBR, CRAM, basic counting, and Weight-Space Programming are significantly more accurate and faster to train on standard reasoning benchmarks like Sudoku-Extreme.

    1 like

  • Alexi Gladstone's profile picture

    @AlexiGlad

    hey! agreed regarding the results, but imo, this is goodhart's law coming into effect, and using non ANN approaches defeats the purpose of these benchmarks sudoku solvers will always be optimal here (we know how to solve sudoku!), so i think testing approaches that won't scale as well (non deep ANN approaches) is somewhat meaningless given that you need to learn everything with a deep ANN, i believe EBR is SOTA!

    2 likes

  • catid's profile picture

    @MrCatid

    Good points I’d like to see people attack harder problems these simple approaches start being worse at

    1 like

  • Jesse Pan's profile picture

    @thefusioncube

    But does it scale…?

    1 like

  • Alexi Gladstone's profile picture

    @AlexiGlad

    we are actively cooking here🧑‍🍳!

    1 like

  • mark's profile picture

    @markshi07

    curious how this transfers to tasks without a clean right answer. does the energy stay meaningful for open-ended generation?