Skills without self-improvement loops are... basically tech debt. This is the process we use on the Warp repo to keep agent Skills useful the more they're run, based on human feedback 👇

We're using this on our issue triage and PR review skills, so they improve the more the community interacts with them. Here's more on how it works:

Spoiler for other juniors: it’s all tech debt.

Turtles all the way down

You pretty much explained what i'm building There is a difference on what I'm building: 1-The agent reviews, not users 2-Agent approves a review with human 3-You earn money for reviewing someone and lifting everyone up A full Agent Economy is inevitable

The feedback loop is the hard part. Most teams collect the signal but never close the loop back into the skill definition. What triggers the rewrite, a score threshold, manual review, or something else?

same thing happens with AI podcast briefings. if the host context doesn't refresh between episodes, the questions get stale before the guest even joins. static briefings decay way faster than you'd expect

agents love appending. pruning is quite rare for me.. does your loop actually cut stale rules anywhere, or does SKILL.md only ever grow?

Agreed. Which is why I have working on

love this

@mattpocockuk

you cannot convince me you're not scott the woz come on

Love this framing. Skills age quickly unless the loop captures where humans correct the agent and turns that into better instructions or tests. Otherwise every repo quietly accumulates prompt debt.

Very Good

So this is not the way people do it currently? Interesting

No one’s doing it!

Well, to change a spec of anything - including a skill - every time anyone makes a suggestion, rather than wait for enough N volume of suggestions with a recurring time T long enough sounds like the logical thing to do Almost feels as if people want to just show iteration cycles as a metric in itself, which is strange Guess Skills should have more rigorous evals so that they doesn’t fall into Goodhart’s vortex

You should name this something - maybe Skill Release Cycle or Skill Iteration Cycle or SDLC where S stands for Skill

I'm surprised that you have to teach this, isn't this like the most obvious thing ever?

Great idea, but how do you collect the feedback? The feedback should come from the user agent that is executing the skill and be stored somewhere where your agent can read it to improve the skill. Do you have a skill for that?

That’s a solid way to keep things from getting stale. Love the idea of skills improving as they get used.

Have the agent categorize the feedback into 3 categories. 1. Anything the agent can fix that doesn’t change behavior. Version release. 2. Behavior change that it can infer the intent of the spec doc. Version release. 3. Anything that needs a human in the loop. Manual review.

a skill that does not learn from its own misses just calcifies, the feedback loop is what separates a skill from a fancy snippet

"Skills without self improvement loops are basically tech debt" is a good way to frame it

Loops without independent reviewers is also basically tech debt. We realized this and built a generic framework for it, haven’t looked at code ever since: