π0.5 vs. MolmoAct2. Same task. Same object placements. Same training data regimen. Precise spatial placement lets us compare models side by side on any matching rollout in our Reality Check benchmark. Someone jokingly called it “Model Smash.” 👇 1/3

Pick a rollout, click “Compare” in our data viewer and let two policies battle it out. Here they are opening a toolbox with a screwdriver. Try it yourself: 2/3

We also try our best to be transparent whenever we fuck up. We’ve added a changelog to the leaderboard so you can see what we corrected—annotations, success rates, or anything else. Spotted a mistake? Please tell us: [email protected] 3/3

will provide valuable insights into model performance.

side by side comparisons make the differences obvious

Have you tried Isaac0.5?

controlled rollouts make robot comparisons far more revealing

paired rollouts with identical placements is the right call, it turns success rate from a noisy aggregate into a per-episode diff. failure-mode tags on the compares (grasp vs placement vs recovery) would make it even sharper, that's usually where two policies actually differ, not the headline %