An open-source tool where Claude Code tunes your AI agent for you: it rewrites the prompts and tools, then measures every change. On one production agent, tested on conversations it had never seen, its builders say it cut failures 2.5x, cost 6x, and ran 2x faster. Repo:

Biggest lesson from letting Claude rewrite my agent prompts: freeze the eval set first and keep it out of its context. Otherwise it quietly tunes toward the test cases and production gets worse while the scores go up.

Agents optimizing agents.

agents tuning agents now, i'm just here for vibes 😭

Let it change one thing per round, a prompt or a tool, never both. When failures drop, you know which edit did it, and when they rise, you can undo just that one. Keep a few hard conversations it never gets to tune on, and score every round against those too.

Finally, “the new prompt feels better” has to survive a blind date with conversations it’s never seen

Another Claude session begins.

2.5x fewer failures on unseen conversations, but did the new prompt hold the cases the old one already passed?

I like that the tuning loop is measured, because rewriting prompts without measuring every change leaves me guessing which edit actually helped. I trust the 2.5x failure cut more than the cost cut, because a cheaper agent that still fails in production costs me more in the end than a slower one that works.

measuring every change is the part most people skip. without an eval, 'improved the prompt' is just vibes

Classic survival strategy right there 😆🤝🚀

2.5x fewer failures on conversations it had never seen is the honest shape. the part that bites later: a prompt tuned against one eval set is an artifact with no provenance, so after the next model swap nobody can tell if a regression is the model or the tuning.