Loading video...

Video Failed to Load

Go Home

FOUR TYPES OF AGENT LOOPS. LOOP ENGINEERING IS PICKING THE RIGHT ONE, NOT THE FANCIEST ONE every loop answers two questions: what starts a run, what ends it hand-run, you answer both yourself, every time 1) turn-based > you prompt, it acts, you review, you prompt again > use...

10,312 views • 24 days ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

HOW TO USE AI LOOPS TO RUN YOUR BUSINESS 24/7 A lot has been written about loop engineering for building products. Almost nothing about using loops to run the business itself. That's the bigger idea. A loop is when you give an agent a goal, a way to check its own work, and permission to keep trying until it hits that goal. Build. Verify. Repeat. Stop when the condition is met. Here's what it looks like in practice: 1/SEO loop You're position 30 for a term you want. The loop runs once a month, makes changes, checks where you rank, and keeps pushing until you're on page one. This is running in production right now on Inbox Zero. 2/Ads loop You're spending $100 a day and losing money. The loop tests creative, checks profitability, kills what fails, and keeps going until the account is in the black. 3/Eval loop Your AI feature is only 88% accurate. The loop keeps adjusting the prompt and swapping the model until it passes 90%. 4/LLM visibility loop People search in ChatGPT now, not just Google. Same loop, new scoreboard. Are we the answer or not? The whole thing hinges on one thing: a metric that comes back black and white. Where do I rank? Did it hit profitability? Did the evals pass? Give an agent that scoreboard and it runs for months. Loops used to run for 30 minutes. These run for a year. Take a step, sleep, wake up next month, take another one. You're basically hiring an agency that never sleeps, gets paid in tokens instead of invoices, and undoes its own mistakes when the number goes down. Full episode on The Startup Ideas Podcast (SIP) 🧃 watch

GREG ISENBERG

82,714 views • 1 month ago

your agent loop needs 8 exits. most people ship only one. (explained with triggers) 1) goal met → an evaluator scores the output against a rubric, and the run stops on a pass. → fires when the work is measurably done, not when the model says it is done. 2) turn cap → a hard ceiling on iterations, counted and enforced by the harness, not the prompt. → fires on the task it was never going to finish, before you pay to find that out. 3) budget cap → a limit on tokens or dollars, whichever one runs out first. → fires mid-run, which is exactly why it is the exit that saves you the 3am bill. 4) wall clock → a deadline on elapsed time, independent of how much progress was made. → fires when the run collides with a deploy window or the start of business hours. 5) no progress → hash the state every turn and compare it against the last few. → fires when three turns in a row change nothing. busy is not the same as moving. 6) human interrupt → an approval gate before risky steps, plus a kill switch that lives outside the loop. → fires whenever you decide, and it is the one exit the model cannot argue with. 7) error threshold → a counter of consecutive failures that resets on any success. → fires at n in a row, so it halts instead of retrying into the same wall all night. 8) external event → a webhook or a poll on whatever the task was actually about. → fires when the PR merged or the ticket closed and the work stopped mattering. a loop with one exit hangs. a loop with eight is a system. write the exits before you write the prompt.

Hanako

181,713 views • 1 month ago

10 agent evals for AI engineers: (explained with usage) 1) golden set → a fixed set of cases you never edit, run on every single change. → use as the baseline that tells you whether anything moved at all. 2) llm as judge → a second model scores the output against a written rubric. → use when the answer is open-ended and there is no string to match against. 3) rubric scoring → one number per dimension: correctness, tone, safety, cost. → use when a single score hides which part actually got worse. 4) trajectory eval → grade the path the agent took, not only the answer it landed on. → use when the right answer for the wrong reason is going to bite you later. 5) tool unit tests → test each tool on its own, with fixtures, no model in the loop. → use always. most agent bugs are tool bugs wearing a costume. 6) regression suite → replay past runs against the new prompt or model and diff the results. → use before every prompt change, because prompts have no type system. 7) a/b in prod → split live traffic between two versions and compare outcomes, not vibes. → use when offline scores stopped predicting what users actually do. 8) human review → sample a slice of runs and have a person grade them honestly. → use to calibrate your judge, because a judge nobody checks quietly drifts. 9) shadow run → the candidate runs on real traffic in parallel and its output is shown to nobody. → use before a risky rollout, when one bad answer would be expensive. 10) red team → deliberately attack it: jailbreaks, injection, exfil, tool abuse. → use before anyone external can reach it, not after. offline evals tell you it works. online evals tell you it still works. both sides matter, but not all ten do. run the two that would have caught your last outage. save this. then read the full breakdown on loop engineering below.

Hanako

150,676 views • 1 month ago

You Can Learn AI Agent Harness & Loop Engineering In 19 Min, with LLM Ops, Eval, Tracing and RAG. They went viral not because they're complicated but because they're simple building blocks, and once you see them you can prompt your way to building real systems. 🎬YouTube: Here's the whole thing in one picture. An LLM is a powerful brain that knows everything about humanity and nothing about you or the software you're running. The harness is the set of tools you put on that horse so it runs where you want. Memory gives it context: who you are, what happened before, how to act. The loop lets it call tools again and again, with guardrails so it knows when to stop. Eval and LLM Ops trace every run, score it, and feed the fixes back in so the system keeps improving itself. Master these four and you can read almost any AI agent repo or paper and actually know what's going on. You Can Build Anything. You Can Learn Anything. 💪 Chapters: Intro: the 4 AI agent buzzwords What an AI agent run actually is The memory system: procedural, semantic, episodic What "harness" really means (the horse) Storing and updating memory (databases, skills, summarizer agent) Retrieval: RAG, SQL vs semantic search Tool calling and why agents loop Loop engineering and end-loop guardrails A Claude Code hooks example Eval and LLM Ops: why you need them Tracing every run (Langfuse, LangSmith) Evaluation: LLM as a judge Diagnosing what broke The gate: ship the fix or fix the bug Zoom out: the full system

Shen Sean Chen

15,952 views • 2 months ago

Loops vs. Graphs, clearly explained! loops are great, but they have a ceiling: a loop makes one unit of work better. it cannot decide which units exist. so you end up with a very good agent running the wrong three steps, in the wrong order, one at a time. Graph engineering fixes this by moving the decision up a layer: what runs, what runs at the same time, and what never runs at all. you need both. here's how it works: a graph splits your system into two kinds of decision. ↳ inside a unit: the loop. produce, check, correct, repeat until green ↳ between units: the graph. split, fan out, merge, gate, send back Prompts → Context → Harness → Loops → Graphs you get parallel work, isolated contexts, and steps that stop running when nothing needs them. the trick is being selective about what becomes a node. only spend a model where judgment lives. merging, ranking, deduping and schema checks are edges, and edges are code. free, instant, and they cannot be argued out of a verdict. a graph where every edge is an agent pays rent on its own wiring. one thing to know before you scale it. a graph has two return paths, and almost everyone builds one. ↳ the correction edge is short. a gate rejects one unit back to the step that produced it, and it fixes the run you are in ↳ the learning edge is long. an accepted result goes back to the splitter as a constraint, and it fixes every run after skip the second and you get a graph that is fast and never gets smarter. next week it starts from the same place with the same blind spots. and a smaller one that eats whole nights: when a unit fails, return that unit, not the batch. send back four slices because one failed and you have just rewritten three correct ones. do it twice in a run and the run never converges. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓

Hanako

65,507 views • 5 days ago