Video wird geladen...
Video konnte nicht geladen werden
Today we're introducing Web Skill Factory: a pipeline that turns solved web tasks into reusable, verified code. Most agent "skills" are notes a model rereads and reinterprets every run. Ours are programs. On a WebArena subset with gpt-5.4, reusing the library moved accuracy on held-out instances from 55% to... show more
31,189 Aufrufe • vor 2 Monaten •via X (Twitter)
13 Kommentare

Every task a web agent solves leaves a working script behind. Web Skill Factory distills those scripts into a growing library of parameterized skills: code that runs without a model and composes into future tasks, instead of re-exploring a site from scratch.

A learned skill is a standalone CLI program. It re-executes in about 40 seconds with zero tokens. Recurring tasks can just be scheduled rather than paying a model to redo the same work.

Skills generalize past a single lucky run. We align multiple solves of the same task template, keep the shared workflow as the program skeleton, and lift the differences into parameters. That's what stops a skill from being correct but overfit to one instance.

Two gates before anything enters the library. The original solve is checked against a gold answer when one exists, self-verified when it doesn't. Then the distilled skill has to replay standalone. Broken or non-executable skills don't get in.

Skills also evolve as they're reused. New solves widen an existing skill in place, and regression replay confirms later updates haven't broken behavior that already passed.

Built on WebWright. -

Work by @demisama_ and @Adamlu28 🚀

agree with skills-as-programs over skills-as-notes — I've seen the same distinction in my harness work: a skill that carries verification (what 'done' means, how to check) is reusable; one that's just context text gets reinterpreted differently every run. the 55→70 on held-out instances is exactly the delta I'd expect. verified skills are the unit of compounding for agents.

55% to 70% accuracy just from reusing skills as code is a big jump

Web Skill Factory turns solved web tasks into reusable code

Turning successful web tasks into reusable programs is a major upgrade

Turning solved web tasks into reusable verified code instead of notes is brilliant

Code not notes is the leap agents actually needed
