Sharing our work at NeurIPS Conference on reasoning with EBMs! We learn an EBM over simple subproblems and combine EBMs at test-time to solve complex reasoning problems (3-SAT, graph coloring, crosswords). Generalizes well to complex 3-SAT / graph coloring/ N-queens problems.

To solve complex reasoning tasks, our approach formulates the reasoning task as optimizing the summed energy functions over EBMs learned over subproblems of the task.

To effectively optimize this composed energy function, we propose a parallel optimization process where we jointly optimize a set of particles at once.

Our approach is able to generalize well to complex reasoning tasks -- outperforming specialized SAT solving methods on 3-SAT problems such as NeuroSAT and NSNet.

More details and illustrations can be found at the webpage here: and code at

@NeurIPSConf Brøther u know I love this kinda work. Seems like you focused on discrete problems here. Did you explore at all how the continuous relaxation you used impacted the performance of learning or sampling ?

@NeurIPSConf Reasoning with EBMs seems very applicable to robotics!

Love this work. One question: how does this handle noise filtering tasks where most input is irrelevant? Your energy composition excels at structured reasoning (SAT, graph coloring). But what about tasks requiring selective attention - finding rare signal in long noisy sequences? Different architectural principle needed there?

@NeurIPSConf Good question! In this setting, instead of optimizing the sum othe energy function, you could instead optimize the softmin or K lowest energy values. This would allow you to selectively attend to rare signals.

@NeurIPSConf How does the optimization work? And what size problems can it solve?

@NeurIPSConf We run gradient descent on a set of particles and periodically resample them. The method works well for problems with 8-20 models composed.

@NeurIPSConf Hi thanks — by model composed, do you mean sub-problems? Why can’t it be generalized to any combinatorial problem? Specially if you have thousands of various boolean constraints?

@NeurIPSConf Yes, subproblems. You can in principle also have thousands of constraints -- the optimization problem across constraints will just be hard then.

@NeurIPSConf any thoughts on how the compositions of energy functions could be learned instead of hardcoding? that seems important outside of combinatorial optimization (for example, if you wanna model natural language)

@NeurIPSConf Yes, you could definitely learn the composition! Each energy function could be conditioned by a latent inferred by an encoder, which could then allow you to learn the composition

@NeurIPSConf interesting, do you mean using those latents from an encoder as a soft routing among (or some weighted combination of) energy functions?

@NeurIPSConf Yup!

@NeurIPSConf okay so similar to MoE but each expert has an explicit energy function? curious how end to end training would work when you have multiple energy functions at different levels, is there any paper that does this?

@NeurIPSConf Yes, it would be something like or though all energy functions are at the same level.

@NeurIPSConf Interesting work. Curious, do you have a sense of how the difficulty of training the EBMs on subproblems scales on tasks with larger input spaces?

@NeurIPSConf It depends on how difficult the subproblem is -- the harder the subproblem is, the harder it is to train the EBM. The input space size doesn't matter that much.

@NeurIPSConf NIODOO's TQFT acts as a reasoning engine for transitions, while EBMs focus on state-based energy landscapes. They complement each other, with potential synergies in hybrid models.

Energy-Based Models sound fancy, but think of them like chefs balancing flavors; they learn which combinations of features taste right for the task. Additive EBMs? That’s when the chef adds each ingredient thoughtfully, so the dish can reason about each flavor instead of throwing everything into one mystery stew.