We gave frontier AI coding agents 24 hours to write a complete Game Boy Advance emulator from scratch. GPT-5.5's emulator runs games best, with Claude Sonnet 4.6 and Opus 4.7 close behind. Gemini 3.1 Pro failed to produce a working emulator.

Here's GPT-5.5's emulator running Goodboy Galaxy side-by-side with Mesen2, a widely-used open-source GBA emulator we grade against. Given the same sequence of button presses, the two play out nearly identically frame by frame. More at:

Here's Claude Opus 4.7's emulator running Collie Defense. The rendering differs from the reference in places, but the game is playable, scoring 89% on video. The audio doesn't work, which is true for most games on Opus 4.7's emulator. Video and audio are graded separately.

Claude Sonnet 4.6's emulator gets Celeste mostly right, scoring 71% on video. You can play the game, but the main character flickers in and out. Play games on AI-built emulators here:

By contrast, Gemini 3.1 Pro does not produce a working emulator at all. Its emulator compiles and runs, but produces only blank screens or visual noise. The same is true for Kimi K2.6. GLM 5.1 and MiniMax M2.7 fare even worse, producing emulators that don't compile.

Here's how each model scored over 24 hours of wall-clock time, on a single attempt. GPT-5.5 performs best, with a late gain in the final hours. Claude models settle in by hour 8 and cluster within noise. More at:

GBA Eval grades emulators on three things: how they handle real gameplay across games, audio fidelity, and open-source tests for hardware behavior. Gameplay carries the most weight. If audio breaks, you can still play with the sound off. If video breaks, you can't play at all.

You can try these emulators at where you can pick any model and any game and play them in the browser. How we grade replays, how we designed the environment, and what different scores look like: Code:

The Game Boy Advance has no entropy source. In-game RNG comes from the precise frame timing of inputs, which is how speedrunners can manipulate random outcomes by timing their inputs exactly.

GBA Eval uses this to grade. The same inputs produce the same gameplay on any correct emulator, so we can replay a sequence frame by frame against a reference. It's a single, unusually verifiable task. Most software doesn't have that property, which is what makes evals hard.

Mechanize's commercial work uses similar grader and environment design, but the tasks themselves usually look different. Models are already good at pure coding, so the interesting bottlenecks are ambiguity, edge cases, and realistic testing.

We don't usually share details about our commercial work. We're releasing GBA Eval to give people a sense of what we work on:

If you'd want to work on problems like this, we're hiring:

Why give agents 24 hours each rather than (say) $500 of tokens each?

This is the right shape for coding-agent evals. Not tiny tasks with clean answers, but long-horizon systems where bad architecture compounds. Emulators are great because the bugs have nowhere to hide.

cool work. mgba era is still unsolved but you can give this paper ( as a markdown skill for minimizing sim-to-sim gap. It worked with the pyboy gameboy emulator to convert to Rust

the reason most people don't understand the gap between gpt's capabilities and real-world performance is that they're thinking about the code, not the actual problem being solved

Building a Game Boy Advance emulator in 24 hours is a genuinely hard benchmark, it requires cycle accuracy and hardware timing knowledge that can't be faked. Gemini failing entirely is more telling than GPT winning.

Looks like AI is about to be loved by the game preservation crowd

wow

GPT 5.5 best on GBA emulator from scratch with Claude Sonnet and Opus close behind

very cool benchmark you should definitely grade them on framerrate / performance compared to baseline emulator too