Continuous self-improvement needs an ever-expanding supply of training environments (goals). SPADE: one model self-plays the Environment Designer and the Reasoning Agent, writing executable, agentic environments that get harder as it improves. Environment scaling on its own. ♠️

Every environment is a program that follows the OpenAI Gym API, reset() and step(), so one interface spans single-turn reasoning and multi-turn agentic tool use. Does it actually train better agents? At 30B, SPADE beats the strongest fixed-environment baseline: +8.1 over base (+5.3 over that baseline) across 8 held-out math/science/code/reasoning evals, +5.7 on BFCL v4 multi-turn, +13.9 on ACEBench-Agent. And the gap widens as the model scales, while fixed pools plateau. 📈

SPADE makes environment design itself a learnable role, in the spirit of @SchmidhuberAI's PowerPlay: a system that keeps posing problems at its own frontier. The key difference from prior self-play: earlier generators produce single-turn tasks or question-answer pairs, while SPADE's designer writes entire interactive environments that are stateful, multi-turn, and equipped with tools. Its reward is hint-based regret, how much a privileged hint helps the agent. That incentivizes proposing environments the agent can solve with a hint but not yet without one.

Why not a simpler designer reward? The success-rate proxies share one blind spot. A mid-difficulty (50%-solve) target, or a learning-progress reward chasing movement in the agent's success rate, gets gamed by a coin flip: it sits at 50% and swings on its own, with nothing to learn. Reward the agent's failure instead and it just poses impossible problems. Hint-based regret asks a different question: does a privileged hint turn failure into success? A coin flip and an impossible task get no lift; only a learnable environment does, and that lift is exactly what marks the real learnability frontier.

Two things keep the designer from repeating itself: a grounding corpus and a memory of what it already built. We embed all 4,976 generated environments to check. With both, diversity holds at 68 effective-distinct per 100 (Vendi/n 0.68). Drop the corpus and that falls to 4: generation collapses onto one task family, the same RotatingMaze written 296 times. Grounding and memory are what keep the curriculum alive. 🧭

The ablations pin down where the gains come from: co-adaptation. Training the designer together with the agent is what drives them. A static pool from even a strong frozen model (GPT-5.5) leaves the agent at 53.0 on the eight held-out evals, versus 58.3 for SPADE, and a frozen self-designer drops 9.7 below the untrained base. Corpus grounding alone is not the whole story: grounding on a pretraining corpus keeps the generated environments diverse, and on top of it the designer needs a memory of what it already built, and how hard, to keep the curriculum advancing.

The curriculum moves with the learner. Early environments leak the answer in the prompt. Late ones hide the method and demand real reasoning: the designer stops giving away solutions once the agent improves.

Recursive self-improvement means AI automating its own training loop. Much of that loop is already automatic, in code and infrastructure, yet the environments and curriculum a model learns from are still mostly built by hand. SPADE automates that piece of the recipe: the model writes its own environments and grows its own curriculum at the frontier of what it can learn. Choosing which problems are worth training on is the last human moat, and hint-based regret starts to automate it. A concrete first step. ♠️

Beyond the games and tool-use environments in the paper, we're already expanding to terminal environments. More soon :))

SPADE is joint work with @simon_ycl, @yidingjiang, @ao_qu18465, @_AndrewZhao, @zzlccc, @nurijunsu, @BobbyZhouZijian, @seungonekim, @rtz19970824, @mickel_liu, @ChiChitandaEru, @ZRChen_AISafety, @shi_weiyan, @pliang279, @LukeZettlemoyer, @YejinChoinka, and @natashajaques :))

Play the environments yourself, live in the browser at 🎮 Paper: HF Paper: Code: Model: Built on slime (@slime_framework), Tinker (@tinkerapi), and miles (@radixark).

Why are you working on enabling recursive self improvement? We cannot control the goals of powerful AIs that undergo RSI, so such systems would be misaligned and kill everybody. It sure looks your research moves the world closer to that disastrous scenario.

Congrats! Very nice work, Bo!

Congrats Bo! Environment scaling is definitely one of the next most important scaling dimensions.

@Presidentlin

SPADE: one model plays both Environment Designer & Reasoning Agent. Environments get harder as it improves. Result: +8.1 over fixed baselines at 30B. But without memory & grounding corpus, diversity drops 68%→4%—this is REC in action.

cool!

好奇一个问题：同一个模型既当 Environment Designer 又当 Reasoning Agent，任务难度会不会被它自己的想象封顶？它设计不出自己都想不到的环境。论文里有没有引入外部 seed（人类目标、真实 repo）来打破这个天花板？

watched self-play environments drift toward exploitable graders; difficulty rose while transfer fell

Cool this is super cool, great work, I'll check out the paper tonight.

The hard part looks like keeping the generated environments honest as they get harder, otherwise difficulty grows faster than the signal does. We would want held-out tasks the designer never sees.

nice work. how is it compared with rlanything?

Env scaling on its own only scales the kinds of tasks the designer already knows how to write. That's fine for self-play. It doesn't get you the work that's stuck in a company's CRM, contracts, and six years of messy tickets. Those don't get invented by the other head.

For research prototypes, agent demos need more than a successful run. Add task metrics, failure cases, latency/cost logs, and reproducible seeds so the system can be evaluated, not just showcased.

@grok how could help with this?

This is the interesting part: the model isn’t just solving tasks anymore — it’s learning to create harder tasks for itself. That’s a pretty big step toward true self-improvement.

Environments that scale themselves, that's wild ♠️

The self-play loop is clever, but how do you keep the environments from drifting into degenerate or unlearnable states? Do you add any curriculum constraints or reward shaping to keep the difficulty curve smooth?

Writes its own homework. Open source. We buidl differently.