Introducing PROWL! We’ve built RL agents that explore game environments, tasked with discovering failures in world models across physics, visuals, and actions. Those failures then become training data in an automated loop that advances world model performance.

In Minecraft, PROWL unlocked major improvements in action following, visual quality, temporal coherence, environmental dynamics, and out-of-distribution robustness. It even learned to simulate entirely new behaviors discovered by RL agents.

We see PROWL as another lever of scale for world models, which we're excited to explore further. You can find our technical summary and paper here, and please let us know any questions or thoughts.

Self correcting world models trained on their own failures is one of the most elegant feedback loops in AI right now.

We agree.

4/ The other piece is the a prioritized memory of discovered failures. After each WM update, every trajectory gets re-scored. Solved cases decay out, unresolved ones rise. It's not a replay buffer. It's a self-adjusting curriculum over the model's current failure frontier.

So excited to see this work from the team starting to bring more of the tools that have worked so well with LLMs into the world model space.

this will be huge.

1/ Really proud of what the team pulled together on this one. Here's what we developed to make PROWL work 👇

7/ We see this as a next lever for scaling world models — and for the intelligent agents that learn inside them. Better simulators make better agents. Better agents find harder failures. This is what open-ended learning should look like.

5/ Honestly the most exciting result for us: PROWL discovers 27 strictly novel action compositions — combinations that appear nowhere in the human dataset or the non-adversarial baseline.

3/ That single design choice splits two regimes cleanly: 🔴 Weak anchor → reward hacking, zero novel behaviors discovered 🟢 KL-anchored → finds failures that are hard AND behaviorally valid Novelty without learnability is useless. You need both.

2/ The key insight: prediction error alone isn't a good reward. A weakly constrained adversary just learns to thrash the camera to reward hack — the WM is genuinely wrong, but the failures don't generalize. So we anchor the adversary to a behavioral prior with a KL constraint.

@ahguzelUK is the 🐐 for sure

This is the missing ingredient for world models.

8/ Huge thanks to the entire team for making this happen. Now let's push the field forward 🚀

6/ And the curriculum is real. Short-horizon improvements compound through autoregressive rollout — long-horizon gains where the matched-compute baseline barely moves. Better local dynamics → better long-horizon prediction. Which is the whole point.

Failure-as-training-data, automated. The product-design analog: most startups have a feedback loop measured in weeks. The winners compress it to hours. PROWL is basically Antifragile-as-a-service for world models. In agents, the loop IS the product.

super cool!

❤️

Agents that search for failures are more interesting than agents that only complete tasks. A discovered crack in a world model is a training asset. The loop is not demo → wow; it is explore → break → turn the breakage into data.

Interesting to know what was framework used for rl training - was it something like verl?

our fewture fresn 'll be ai aint

Breaking physics to fix them? Genius move.

why is this cool ?

sounds intense. hope the training loop is efficient; otherwise, it could get chaotic fast.

That was fast - this looks awesome.

so it's a GAN for world models. adversarial training is famously stable and never collapses into a local minimum. right?