Mechanize's banner

Mechanize

@MechanizeWork • 13,923 subscribers

We build environments and evals for training and evaluating frontier coding agents.

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

We gave frontier AI coding agents 24 hours to write a complete Game Boy Advance emulator from scratch. GPT-5.5's emulator runs games best, with Claude Sonnet 4.6 and Opus 4.7 close behind. Gemini 3.1 Pro failed to produce a working emulator.

We gave frontier AI coding agents 24 hours to write a complete Game Boy Advance emulator from scratch. GPT-5.5's emulator runs games best, with Claude Sonnet 4.6 and Opus 4.7 close behind. Gemini 3.1 Pro failed to produce a working emulator.

94,441 Aufrufe • vor 1 Monat

We evaluated Gemini 3.5 Flash on GBA Eval. It could not build a working GBA emulator. On Piugba, the game just flashes on screen, unplayable and with no sound. Overall, it achieves a score of 6.7%.

We evaluated Gemini 3.5 Flash on GBA Eval. It could not build a working GBA emulator. On Piugba, the game just flashes on screen, unplayable and with no sound. Overall, it achieves a score of 6.7%.

52,287 Aufrufe • vor 26 Tagen

If someone from the future gave us a trillion tokens of agents competently planning, executing, and adapting, we'd likely turn today's models into highly capable agents overnight. But we don't currently have that data. We at Mechanize are working to solve that.

If someone from the future gave us a trillion tokens of agents competently planning, executing, and adapting, we'd likely turn today's models into highly capable agents overnight. But we don't currently have that data. We at Mechanize are working to solve that.

27,890 Aufrufe • vor 10 Monaten

Keine weiteren Inhalte verfügbar