Loading video...

Video Failed to Load

Go Home

Every open-source project should be engineering agent loops right now. We've found success managing Warp with loops for: - Issue triage - Spec generation for larger features - Code review - Even letting agents self-improve their Skills on a cron

41,258 views • 4 months ago •via X (Twitter)

25 Comments

Ben Holmes's profile picture
Ben Holmes4 months ago

If you want a template, we also open-sourced our management framework. Still early and would love feedback on it: If you wanna set it up on your project, DM me! We can help with the token cost :)

Andrei Maxim's profile picture
Andrei Maxim4 months ago

@warpdotdev That's not a loop. A cron job, callback, event listener are more accurate terms than "loop". Based on what you describe, you invoke the triage skill (or whatever skill you want) once per PR, hence not a loop.

Ben Holmes's profile picture
Ben Holmes4 months ago

@warpdotdev Fair, expanded here

Andrei Maxim's profile picture
Andrei Maxim4 months ago

The two loops described there are not loops: one is a callback and the other is a cron job. I don't think anyone would name either of those as a "loop". You could argue that it's a "loop" as in "feedback loop", but that case you'd need the output to be redirected back to the system, but this is not the case in either of those examples. I know it sounds like I'm nitpicking, but I truly do think that words matter and being accurate matters, especially when we live in a world where the natural language we use can produce significant chunks of code.

Ben Holmes's profile picture
Ben Holmes4 months ago

@warpdotdev Eh, fine. I’ll agree to that

Akshay's profile picture
Akshay4 months ago

@warpdotdev What exactly is the “loop” here ? Trying to understand how this is different from a multi agent system.

Ben Holmes's profile picture
Ben Holmes4 months ago

@akshaydeshraj @warpdotdev The part that makes it a loop for us is self-improvement. We set up a few agents on schedules to evaluate GitHub comments on issues and PRs to see if our automations Skills can be improved. @zachlloydtweets expanded on our setup here:

Clifton Cunningham's profile picture
Clifton Cunningham4 months ago

@warpdotdev I couldn't agree more! Here is mine for my open source projects (it is managing itself, and spec public):

Will Washburn's profile picture
Will Washburn4 months ago

@warpdotdev yes, absolutely. I think the goal for OSS projects should be to make your repo slop proof by having a bunch of loop systems that correct / review / improve contributions

Max Howell's profile picture
Max Howell4 months ago

@warpdotdev You should let it loop to remove all the bloat

Yunfei He's profile picture
Yunfei He4 months ago

@warpdotdev /loop is the way.

Phi Browser's profile picture
Phi Browser4 months ago

@warpdotdev Agent loops are the right direction. What makes them safe to actually let run is observability: watching the agent work, not just reading the diff after. The teams that win give the human a live view and a way to step in, not just an output to approve.

Dom Hallan's profile picture
Dom Hallan4 months ago

@warpdotdev You lost me at "every" because there is few scenarios that realistically fit EVERY project.

dwi budi's profile picture
dwi budi4 months ago

@warpdotdev What actually warp is? Coding agent or terminal? Drop a beginner video tutorial please.

Ben Holmes's profile picture
Ben Holmes4 months ago

@warpdotdev Warp is a terminal to see what agents are doing, Oz is our automation platform to host agents on cloud runners if you need it. I made a getting started playlist on both here!

Ralf Kronen's profile picture
Ralf Kronen4 months ago

@warpdotdev Most of those loops have a judge downstream. Triage, spec gen, a human reads it next. Code review is the odd one out: it grades the same kind of work that wrote the diff, and self-improving on a cron just tightens that circle. Which loop has a check the agent doesn't own?

Chungus's profile picture
Chungus4 months ago

@warpdotdev

Mohamed Abdelgwad's profile picture
Mohamed Abdelgwad4 months ago

@warpdotdev Hey Ben, what tool are u using for your videos? Can it work with streaming too?

Ben Holmes's profile picture
Ben Holmes4 months ago

@warpdotdev @screenstudio !

Waves's profile picture
Waves4 months ago

How are you measuring issue triage success rate? Its very common for teams to say “agent easily triaged a b c!!!” Based off a few cherry picked examples But never really scales once you are triaging 100s per day and it’s pretty easy to loose customer trust when your agent is wrong

Utsav Ladani's profile picture
Utsav Ladani4 months ago

@warpdotdev My team is starting a new project. Looking forward to set this up for it, especially skills of this GH repo.

xeno_+'s profile picture
xeno_+4 months ago

@warpdotdev you guys aren't seriously just starting on loops now, are you? I don't think I believe that.

Ben Holmes's profile picture
Ben Holmes4 months ago

@warpdotdev Been doing it for months. Just using the vocab that's hip with the kids

Timur Yessenov's profile picture
Timur Yessenov4 months ago

@warpdotdev This is the loop shape I trust: the agent changes the repo, then later reads real GitHub comments and updates the skill. The missing metric I’d track is simple: did the next run make the same mistake less often?

Fer's profile picture
Fer4 months ago

A lot of open source projects do not use github, they use gitlab, gitea, codeberg, sourcehut, cgit and lots of other fully proper open source solutions for git so this is sadly not useful for a lot of older OSS projects, especially ones in the linux/bsd space. It would be nice to see this but a bit more vendor agnostic.

Related Videos