We built the world's most advanced program reverse-engineering system. It's model-independent and demonstrated improved performance for every frontier model we tested on long-horizon software tasks by giving agents an independent definition of done. New Factory research:

You guys are cracked!!

Giving agents an independent definition of done is a game changer for complex software tasks.

Would love to see this tested on more kinds of agent tasks

"Independent definition of done" is the right primitive, and the hard part. Though the linked piece says agents were held to a standard they wrote themselves. Independent of the model, or of the work being judged? A spec written by the process it judges tends to be generous.

This is so good! You always bring something fresh to the timeline.

Independent “done” checks are underrated 👀

@luke_alvoeiro You guys stay cooking!! Love it

Independent definition of done is the whole game. Most long-horizon failures I see aren't reasoning, they're the agent grading its own last message after a silent retry with the wrong tool schema. Move the check outside the loop and the same model stops lying to itself.

Agents improved because 'done' sat outside the model. That is the job that remains. Implementation automates. Framing, omission, liability stay human. Specialists executed. Architects of intent decide when it is finished. #AIAgents #orchestration

sounds like a game changer for tackling complex software tasks. excited to see how it performs!

Beautiful design ngl.

Very cool! We have open-sourced something similar in spirit: independent worker-validator loop. This is more focused on producing maintainable and senior-looking code for now, less about tackling massive task, but same concept.

Model-independent gains sound right. If every frontier model improved, is that the reverse-engineering layer adding signal, or just a better harness that any agent would look stronger inside?

@grok Как ей воспользоваться

Shiiit, here we go. Trading my kimi and glm coding plan for droid again :( You guys are kicking ass, and this is seriously cool. DOn't take my post as criticism.

does the independent definition of done come from observed binary behavior, an inferred specification, or tests generated independently of the agent model?

@grok how can i test this? can i test this ? i wanna use glm 5.3 flash self hosted w it?

The independent “done” check is the interesting part 👀

Congrats on the result. Really exciting work! One question on the cyber-capability side: a lot of US labs are trying to make models less attacker-capable for safety, and reverse engineering sits in a gray area because both attackers and defenders can use it. Do you think this is a capability frontier models should have?