🚀 How should LLMs sample on hard reasoning problems during post-training and inference where direct rollouts rarely produce a correct answer? Best-of-N (e.g., GRPO) and tree search share two limitations: 🔻 Verification signals are sparse 🔻 Candidates stay within the model's own distribution We introduce BES: Bidirectional Evolutionary Search... — a search framework that couples forward candidate evolution with backward goal decomposition. ✅ Works for both post-training and inference.show more

1/8 Why current LLM search hits a wall on hard problems: 1️⃣ Sparse signals: binary success/failure only at the end 2️⃣ Confined exploration: autoregressive expansion keeps every candidate inside the model's own distribution (we prove this is a narrow "entropy shell" later). Hard answers tend to live outside this shell, and exploration is needed to reach them.

2/8 🪄 Forward search in BES adds 4 evolution operators that recombine trajectories: 1⃣Combination: concat suffixes of two paths sharing a prefix 2⃣Deletion: drop an interior step 3⃣Translocation: transplant a step from another path 4⃣Crossover: splice prefix of A with tail of B

3/8 🎯 Backward search recursively decomposes the task into a tree of checkable sub-goals. A forward candidate is scored by how many sub-goals it covers. The signal stays informative even when no candidate has solved the entire problem yet. Sparse end-of-trajectory feedback now becomes dense intermediate guidance.

4/8 🔄 Putting it together: forward and backward feed each other. Backward scores each candidate by sub-goal coverage, favoring complementary pairs (two candidates that together cover more of the goal tree). Forward search then uses evolution operators to recombine those complementary pairs into a candidate that covers more subgoals. Both come with theoretical motivations: evolution escapes the entropy shell, and backward signals exponentially reduce the candidates needed.

5/8 Post-training results. On hard reasoning tasks where existing post-training algorithms fail to make clear progress, BES achieves steady, consistent improvements.

6/8 Inference results. BES achieves better results than existing open-source frameworks on open problem solving tasks.

7/8 BES is fully open-source! 📄 Paper: 💻 Code: 🤗 Trained models: 🌐 Project page:

8/8 Huge thanks to my amazing coauthors @ZhentingQi @Huangyu58589918 @Weirui_Ye, Himabindu Lakkaraju, @ShamKakade6, I've learned so much from you. Special thanks to my advisor @du_yilun for his guidance throughout this work. If you have any questions or feedback, my DMs are always open.

Nice video! How did you make that?

Thanks for your words! I actually made it using Claude Code. First I used the website design tool to make a website, and then ask it to convert the website into a video. It still needs some sort of human intervention, but overall it’s very convenient.

This paper of mine might be interesting for you: Bidirectional Planning in Human Decision Making.

Sounds interesting and related! I will include this a related work in the next version.

Congrats on the new work! So productive.

Thanks Siqiao! Reading your work has always been enjoyable. 😊

Interesting approach, great work!

牛大福

It’s my fortune to learn from Johnson!

great, i like this video

This makes a lot of sense - backward induction is how we solve a lot of problems as well! Have you compared a fully backward induction technique vs a fully forward search technique to see which one contributes more? (I think the ablation studies never do fully backward if I"m reading it correctly)

Hi, thanks for the comment! Only using backward induction to solve a problem would be interesting. However, in our current setting, the backward search only generates subgoals instead of providing a solution, so we cannot do "a fully backward induction".

Ah gotcha - yeah I think it would be interesting to try! I suspect I might work better than forward search because there's likely a "funnel" that converges on the right solution so that search space stays smaller for longer

bidirectional evolution neat

does the backward composition hold up when subgoals arent cleanly verifiable?

Hi, thanks for your attention! Yeah, the backward composition can still hold up. However, if subgoals are cleanly verifiable, it can significantly speed up the search, so I would recommend trying to make the goals verifiable.

thanks for the thoughtful reply, we're chipping away at a complementary angle - will make sure to cite

does it use open evolve?

Thanks for your attention! We build this on top of ShinkaEvolve

@threadreaderapp please #unroll

为你们感到骄傲!

Interesting and great work! I have a little question: In the backward search stage, how to ensure the final task can be decomposed into several clear sub-goals? For example, give a problem "Here are trajectories of (y| x1, x2...), give a explicit expression to fit y using X".