Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

loop engineering worked for 6 months. then one pipeline forced a graph a loop is one agent on discover → plan → execute → verify, until a stop condition hits enough for most agent work: failing tests, draft + self-review, inbox triage, one research writeup the break is nameable:...

seeconvm's profile picture

seeco

2,697 subscribers

14,534 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

ANTHROPIC ENGINEER JUST SHOWS EXACTLY WHAT GRAPH ENGINEERING LOOKS LIKE WHEN A TASK RUNS THROUGH IT most people arguing about graphs online have never actually watched one execute Task → Researcher → Planner → Writer + Code Agent → Reviewer → Deploy six nodes, one shared state, graph completes itself while you watch loop mode gets disabled the second the task splits into real specialties, one agent stops trying to do everything at once reviewer catches a failure, kicks it straight back to the exact node that broke - not a full restart, no lost context the graph is not a fancier loop - it is the org chart your agents were missing bookmark this and watch it run, then read the article below to see why the timeline just found what production systems already do
0:10

ANTHROPIC ENGINEER JUST SHOWS EXACTLY WHAT GRAPH ENGINEERING LOOKS LIKE WHEN A TASK RUNS THROUGH IT most people arguing about graphs online have never actually watched one execute Task → Researcher → Planner → Writer + Code Agent → Reviewer → Deploy six nodes, one shared state, graph completes itself while you watch loop mode gets disabled the second the task splits into real specialties, one agent stops trying to do everything at once reviewer catches a failure, kicks it straight back to the exact node that broke - not a full restart, no lost context the graph is not a fancier loop - it is the org chart your agents were missing bookmark this and watch it run, then read the article below to see why the timeline just found what production systems already do

leopardracer

53,328 Aufrufe • vor 3 Monaten

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.
0:20

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.

kocer

31,198 Aufrufe • vor 1 Monat

KIMI K3 + OBSIDIAN + LOOP ENGINEERING = A VAULT THAT RUNS ITSELF the core idea: the vault is the loop's state, not the chat window everything Kimi K3 knows lives in a .md file the loop: > capture - a thought lands in 00-inbox > context - K3 pulls links, tags, and neighbouring notes > draft - edits happen inside a git worktree, never the live vault > review - a critic agent checks the diff before anything ships > commit - appended to the vault, nothing gets rewritten the key insight: frontmatter fields like supports, contradicts, and supersedes are graph edges, not metadata - the note format is the write API start with a plain loop, it runs about 2-4x the cost of one direct call > only move to a full graph once state has to outlive the session, several agents need to coordinate, or you have to explain what changed - that jump can run 10-50x one review assistant climbed from 55% to 72% to 84% just by moving through these shapes in order
0:20

KIMI K3 + OBSIDIAN + LOOP ENGINEERING = A VAULT THAT RUNS ITSELF the core idea: the vault is the loop's state, not the chat window everything Kimi K3 knows lives in a .md file the loop: > capture - a thought lands in 00-inbox > context - K3 pulls links, tags, and neighbouring notes > draft - edits happen inside a git worktree, never the live vault > review - a critic agent checks the diff before anything ships > commit - appended to the vault, nothing gets rewritten the key insight: frontmatter fields like supports, contradicts, and supersedes are graph edges, not metadata - the note format is the write API start with a plain loop, it runs about 2-4x the cost of one direct call > only move to a full graph once state has to outlive the session, several agents need to coordinate, or you have to explain what changed - that jump can run 10-50x one review assistant climbed from 55% to 72% to 84% just by moving through these shapes in order

Mr. Buzzoni

10,680 Aufrufe • vor 2 Monaten

AGENT ARCHITECTURE ROUTES WORK. IT DOES NOT REMEMBER WORK. THAT GAP IS WHY YOUR LOOP KEEPS FIXING THE SAME BUG TWICE. these are two different engineering problems. every agent that silently drifts is missing one of them. architecture answers what runs. harness → loop → graph. it defines the tools, the retries, the branching routes, the approval gates. context ops answer what the run knows. write → read → compress → isolate. it defines what gets saved between attempts, pulled in on read, summarized on overflow, and split across sub-agents. for two months i believed a solid harness plus a verifier loop was enough. my coding agent kept re-discovering the same test failure across retries. the loop was working. it just had nowhere to write what it had already learned. here is the decision rule: if your agent forgets across restarts, add write and read. if it stalls on long tasks, add compress. if two sub-agents step on each other, add isolate. architecture without context ops is a well-routed system with amnesia.
0:12

AGENT ARCHITECTURE ROUTES WORK. IT DOES NOT REMEMBER WORK. THAT GAP IS WHY YOUR LOOP KEEPS FIXING THE SAME BUG TWICE. these are two different engineering problems. every agent that silently drifts is missing one of them. architecture answers what runs. harness → loop → graph. it defines the tools, the retries, the branching routes, the approval gates. context ops answer what the run knows. write → read → compress → isolate. it defines what gets saved between attempts, pulled in on read, summarized on overflow, and split across sub-agents. for two months i believed a solid harness plus a verifier loop was enough. my coding agent kept re-discovering the same test failure across retries. the loop was working. it just had nowhere to write what it had already learned. here is the decision rule: if your agent forgets across restarts, add write and read. if it stalls on long tasks, add compress. if two sub-agents step on each other, add isolate. architecture without context ops is a well-routed system with amnesia.

kocer

12,814 Aufrufe • vor 2 Monaten

THE LOOP ENGINEERING SETUP THAT LETS CLAUDE RUN YOUR WORK FOR HOURS WITHOUT YOU you sit there babysitting it prompt by prompt other people set the loop up once and let Claude plan, act, verify and fix its own work with nobody at the keyboard that gap is the whole skill of loop engineering the 3 resources that actually teach it, in the order i'd read them: > Claude's "Getting started with loops" - the cleanest entry point, straight from the Claude team ↳ > Claude loop engineering: how to build an agent that works while you sleep - the full roadmap from a single prompt to a loop that runs overnight ↳ > the Loop Engineering orange book - the deep conceptual breakdown, for when you want the whole mental model ↳ entry point, then practice, then the concept. that order saves you a week learn this and the work that eats your whole day starts running without you
0:12

THE LOOP ENGINEERING SETUP THAT LETS CLAUDE RUN YOUR WORK FOR HOURS WITHOUT YOU you sit there babysitting it prompt by prompt other people set the loop up once and let Claude plan, act, verify and fix its own work with nobody at the keyboard that gap is the whole skill of loop engineering the 3 resources that actually teach it, in the order i'd read them: > Claude's "Getting started with loops" - the cleanest entry point, straight from the Claude team ↳ > Claude loop engineering: how to build an agent that works while you sleep - the full roadmap from a single prompt to a loop that runs overnight ↳ > the Loop Engineering orange book - the deep conceptual breakdown, for when you want the whole mental model ↳ entry point, then practice, then the concept. that order saves you a week learn this and the work that eats your whole day starts running without you

Mr. Buzzoni

18,365 Aufrufe • vor 1 Monat

this is straight f*cking gold the full harness guide for Kimi K3, the #1 open source frontend model the premise: the model underneath keeps changing. the harness is the part that stays yours what one night of it looks like: 02:00 - the trigger fires, every node that needs work gets picked > 212 agents fan out, one per node > a bad return gets rejected, retried once with the reason attached > one agent tries to write outside its folder. blocked. nobody woken up > +41 nodes and +96 edges land in the graph > a drafted email hits pre_send and waits for you 02:41 - the loop stops on its own, inside a 45 minute budget 07:30 - you read one file and make two decisions the whole machine is one folder, one config file, five short scripts two rules hold it together: > the model sits behind one line, so a better model is a config change > the verifier lives outside the agent, so nothing grades its own work now the money part companies burn whole quarters building internal agent platforms a harness setup for a small team goes for four figures, one time then a monthly retainer to run the swap test on every new model release models come and go. the person who owns the harness keeps getting paid
0:16

this is straight f*cking gold the full harness guide for Kimi K3, the #1 open source frontend model the premise: the model underneath keeps changing. the harness is the part that stays yours what one night of it looks like: 02:00 - the trigger fires, every node that needs work gets picked > 212 agents fan out, one per node > a bad return gets rejected, retried once with the reason attached > one agent tries to write outside its folder. blocked. nobody woken up > +41 nodes and +96 edges land in the graph > a drafted email hits pre_send and waits for you 02:41 - the loop stops on its own, inside a 45 minute budget 07:30 - you read one file and make two decisions the whole machine is one folder, one config file, five short scripts two rules hold it together: > the model sits behind one line, so a better model is a config change > the verifier lives outside the agent, so nothing grades its own work now the money part companies burn whole quarters building internal agent platforms a harness setup for a small team goes for four figures, one time then a monthly retainer to run the swap test on every new model release models come and go. the person who owns the harness keeps getting paid

Mr. Buzzoni

13,054 Aufrufe • vor 1 Monat

there are four types of agent loops. most people only know one. loop engineering is a choice between four structures, each handing off one more job than the last. every one answers two questions: what starts a run, and what ends it. hand-run, you answer both yourself, every time. 1) turn-based → you prompt, it acts, you review, you prompt again. both jobs stay with you. use when requirements are still forming. 2) goal-based → "/goal hit Lighthouse 90, stop after 5 tries." an evaluator checks, a no sends it back. use when the outcome is measurable but the path isn't. 3) time-based → a clock fires, it runs "check the PR, fix CI," then waits. /loop local, /schedule survives a closed laptop. use for recurring work. 4) proactive → no human present. it watches a channel, spawns triage, fix, and a reviewer, closes the task itself. use for standing duties you can't predict. not which one is most advanced. whether your task is exploratory, measurable, recurring, or standing. the more you hand off, the less you babysit. full breakdown in the article below.
0:10

there are four types of agent loops. most people only know one. loop engineering is a choice between four structures, each handing off one more job than the last. every one answers two questions: what starts a run, and what ends it. hand-run, you answer both yourself, every time. 1) turn-based → you prompt, it acts, you review, you prompt again. both jobs stay with you. use when requirements are still forming. 2) goal-based → "/goal hit Lighthouse 90, stop after 5 tries." an evaluator checks, a no sends it back. use when the outcome is measurable but the path isn't. 3) time-based → a clock fires, it runs "check the PR, fix CI," then waits. /loop local, /schedule survives a closed laptop. use for recurring work. 4) proactive → no human present. it watches a channel, spawns triage, fix, and a reviewer, closes the task itself. use for standing duties you can't predict. not which one is most advanced. whether your task is exploratory, measurable, recurring, or standing. the more you hand off, the less you babysit. full breakdown in the article below.

Hanako

478,171 Aufrufe • vor 3 Monaten

someone just open-sourced the entire "loop engineering" playbook. for free. 8.6k stars on github for loop engineering not prompts. loops. everything you need to build autonomous AI systems: → scheduling → memory & state → planning → sub-agents → verification → worktrees → MCP → stop conditions → safety guardrails → observability → cost tracking notice something? none of these are prompts. they're the systems wrapped around the model. that's where AI engineering is heading. we're moving from: Prompt Engineering → Context Engineering → Harness Engineering → Loop Engineering soon you'll hear people talking about Graph Engineering too. because once one loop works... the next challenge is coordinating hundreds of them. that's exactly what I explain in: save this. I think "loop engineering" is about to become one of the most important concepts in AI.
0:19

someone just open-sourced the entire "loop engineering" playbook. for free. 8.6k stars on github for loop engineering not prompts. loops. everything you need to build autonomous AI systems: → scheduling → memory & state → planning → sub-agents → verification → worktrees → MCP → stop conditions → safety guardrails → observability → cost tracking notice something? none of these are prompts. they're the systems wrapped around the model. that's where AI engineering is heading. we're moving from: Prompt Engineering → Context Engineering → Harness Engineering → Loop Engineering soon you'll hear people talking about Graph Engineering too. because once one loop works... the next challenge is coordinating hundreds of them. that's exactly what I explain in: save this. I think "loop engineering" is about to become one of the most important concepts in AI.

Rahul

65,527 Aufrufe • vor 3 Monaten

This is pure f*cking treasure. ex-Google engineer reveals his AI agent playbook. 14+ years at Google. $0 to read it. the topic is harness engineering: everything around the model that lets it execute, check its work and recover when something breaks. one example in his breakdown deserves attention. LangChain’s coding agent went from 52.8% to 66.5% on Terminal-Bench 2.0. the model stayed the same. the team changed the prompts, tools and machinery around it. Agent = Model + Harness. the system has six practical jobs: > guides give the agent project rules, constraints and examples > sensors check its work through tests, linters and validators > the loop handles execution, verification, retries and escalation > memory preserves state, artifacts and decisions > permissions control which tools and actions are allowed > observability records tool calls, costs and failures the part worth stealing is what happens after a mistake. an agent keeps declaring broken code finished. you add a verification step before completion. it keeps ignoring a project convention. you add a targeted instruction and a check for that specific failure. it touches files outside its task. you tighten its write permissions. each recurring failure gives you a concrete improvement to make in the harness. that improvement can carry into every future run. this is where the engineering work compounds. bookmark it. start with the mistake you have already paid your agent to make twice.
0:18

This is pure f*cking treasure. ex-Google engineer reveals his AI agent playbook. 14+ years at Google. $0 to read it. the topic is harness engineering: everything around the model that lets it execute, check its work and recover when something breaks. one example in his breakdown deserves attention. LangChain’s coding agent went from 52.8% to 66.5% on Terminal-Bench 2.0. the model stayed the same. the team changed the prompts, tools and machinery around it. Agent = Model + Harness. the system has six practical jobs: > guides give the agent project rules, constraints and examples > sensors check its work through tests, linters and validators > the loop handles execution, verification, retries and escalation > memory preserves state, artifacts and decisions > permissions control which tools and actions are allowed > observability records tool calls, costs and failures the part worth stealing is what happens after a mistake. an agent keeps declaring broken code finished. you add a verification step before completion. it keeps ignoring a project convention. you add a targeted instruction and a check for that specific failure. it touches files outside its task. you tighten its write permissions. each recurring failure gives you a concrete improvement to make in the harness. that improvement can carry into every future run. this is where the engineering work compounds. bookmark it. start with the mistake you have already paid your agent to make twice.

NO1ennn

29,017 Aufrufe • vor 1 Monat

a contractor in Shenzhen priced a ¥12,470,900 hospital contract, about $1.7m, in one afternoon and beat firms carrying forty people he explained how he did it: the bid consultancy he used to pay took three days and ¥46,000 for the same envelope. he did this one alone, off one screen, at 11.4% margin, uploaded before the 17:00 cutoff 214 pages of tender documents read, 68 binding clauses pulled out, 9,485 building parts loaded, 14 places found where a duct and a beam sit in the same cubic metre, deepest one 38mm, all of them fixed, 3,318 lines of quantities priced and the package encrypted and uploaded before the 17:00 cutoff this is Graph Engineering: the job gets cut into small nodes, one narrow task each, wired so that one node's output is the next node's input, and any node is allowed to stop the whole run. it turns a model that answers you into a machine that finishes the job: - give every node one job and one output. a node doing two things fails at both and you cannot tell which one broke - put the cheapest rejection first. his qualification node reads clause 7.4, foreign-owned firms barred, and ends the run four seconds in, before anything expensive touches the model - what moves between nodes is a file. the model travels as a model, the quantities as a table, the price as a number - build exactly one loop: the checker finds 14 collisions, the fixer drops the duct 550mm, the checker runs again, and nothing moves on until the count is zero - cap that loop, or a graph will grind on three impossible clashes until the deadline passes - keep one node whose only job is to say no, and give it authority over everything above it - log each node's output on its own, because when the price comes out wrong you need to know which node believed the wrong thing - run the expensive nodes last, always the catch is that a graph is an extremely confident machine: point it at an outdated rate book and it prices an entire hospital off it without a single node noticing, because no node is asked to doubt the input, only to process it so the nodes that earn their keep are the ones that reject, and almost nobody builds those first bookmark this, the full build with all nine nodes and what each one hands to the next is written out in the article ↓
0:16

a contractor in Shenzhen priced a ¥12,470,900 hospital contract, about $1.7m, in one afternoon and beat firms carrying forty people he explained how he did it: the bid consultancy he used to pay took three days and ¥46,000 for the same envelope. he did this one alone, off one screen, at 11.4% margin, uploaded before the 17:00 cutoff 214 pages of tender documents read, 68 binding clauses pulled out, 9,485 building parts loaded, 14 places found where a duct and a beam sit in the same cubic metre, deepest one 38mm, all of them fixed, 3,318 lines of quantities priced and the package encrypted and uploaded before the 17:00 cutoff this is Graph Engineering: the job gets cut into small nodes, one narrow task each, wired so that one node's output is the next node's input, and any node is allowed to stop the whole run. it turns a model that answers you into a machine that finishes the job: - give every node one job and one output. a node doing two things fails at both and you cannot tell which one broke - put the cheapest rejection first. his qualification node reads clause 7.4, foreign-owned firms barred, and ends the run four seconds in, before anything expensive touches the model - what moves between nodes is a file. the model travels as a model, the quantities as a table, the price as a number - build exactly one loop: the checker finds 14 collisions, the fixer drops the duct 550mm, the checker runs again, and nothing moves on until the count is zero - cap that loop, or a graph will grind on three impossible clashes until the deadline passes - keep one node whose only job is to say no, and give it authority over everything above it - log each node's output on its own, because when the price comes out wrong you need to know which node believed the wrong thing - run the expensive nodes last, always the catch is that a graph is an extremely confident machine: point it at an outdated rate book and it prices an entire hospital off it without a single node noticing, because no node is asked to doubt the input, only to process it so the nodes that earn their keep are the ones that reject, and almost nobody builds those first bookmark this, the full build with all nine nodes and what each one hands to the next is written out in the article ↓

Argona

38,189 Aufrufe • vor 2 Monaten

a developer showed me the cleanest multi-agent architecture i've seen. one mission. nobody steps on anyone else. the mission: ship a safe feature. the graph decides what happens next. router reads the mission and asks one question: where should this go? three workers split in parallel. each in their own private work area. researcher finds the evidence. architect designs it. builder creates it. none of them share context. the three workers run independently. everything lands in shared state. facts, decisions, artifacts. one place where the whole picture exists without anyone copying transcripts. then the crew's work converges. integrator combines what was built. reviewer tests quality and safety. human checkpoint approves anything high-impact. then ship. verified output. private context. shared state. that separation is the whole trick. a prompter asks a question. an architect draws a graph. full breakdown with code in the article below.
0:11

a developer showed me the cleanest multi-agent architecture i've seen. one mission. nobody steps on anyone else. the mission: ship a safe feature. the graph decides what happens next. router reads the mission and asks one question: where should this go? three workers split in parallel. each in their own private work area. researcher finds the evidence. architect designs it. builder creates it. none of them share context. the three workers run independently. everything lands in shared state. facts, decisions, artifacts. one place where the whole picture exists without anyone copying transcripts. then the crew's work converges. integrator combines what was built. reviewer tests quality and safety. human checkpoint approves anything high-impact. then ship. verified output. private context. shared state. that separation is the whole trick. a prompter asks a question. an architect draws a graph. full breakdown with code in the article below.

rvaniaaa

27,465 Aufrufe • vor 2 Monaten

holy sh*t. Jev's founder Diogo Almeida dropped a Google Doc on coding agents one of these PDFs has the map everyone needs: where Jev sits in an agent loop ↓ agents do the work, Jev makes the calls, the LLM only writes: 1 → task in: one Noul before anything runs. in scope, or does a human need to see it first? 2 → dispatch: a Choice picks who takes the step (research, write or review). act at 0.85+, everything below goes to review 3 → act: Jev gates the tool call (allow, confirm or block). the LLM only writes the arguments 4 → check: a Score decides what comes back (drop, summary or keep). whatever Jev is unsure of goes to a human 5 → finish: Jev says done above 0.8, then code proves the file actually exists the LLM shows up in 2 of 5 stages. everything else is a typed answer, plain code or a person. don't refactor your agent. hand Jev the one fork it hits most (usually the tool gate or dispatch) and watch 3 numbers for a week: cost per completed task, time per completed task, and how often escalation fired and whether it was right. if escalation never fires, the threshold is too loose and your agent is quietly running unsupervised.
0:18

holy sh*t. Jev's founder Diogo Almeida dropped a Google Doc on coding agents one of these PDFs has the map everyone needs: where Jev sits in an agent loop ↓ agents do the work, Jev makes the calls, the LLM only writes: 1 → task in: one Noul before anything runs. in scope, or does a human need to see it first? 2 → dispatch: a Choice picks who takes the step (research, write or review). act at 0.85+, everything below goes to review 3 → act: Jev gates the tool call (allow, confirm or block). the LLM only writes the arguments 4 → check: a Score decides what comes back (drop, summary or keep). whatever Jev is unsure of goes to a human 5 → finish: Jev says done above 0.8, then code proves the file actually exists the LLM shows up in 2 of 5 stages. everything else is a typed answer, plain code or a person. don't refactor your agent. hand Jev the one fork it hits most (usually the tool gate or dispatch) and watch 3 numbers for a week: cost per completed task, time per completed task, and how often escalation fired and whether it was right. if escalation never fires, the threshold is too loose and your agent is quietly running unsupervised.

Archive

33,524 Aufrufe • vor 27 Tagen

LOOP RAT: THE TECH UNDER AN AUTONOMOUS CODING LOOP no framework, no service, no database a dozen files of bash and python, plus one per loop one shift runs six phases: > preflight - won't start if state/HALT exists, spend is gone, or the loop's already running > act - runs under a hard timeout. shell gathers facts, the model only judges > verify - runs the plan's verify: command. "tests pass" isn't proof. the exit code is > guard - fingerprints the tree. blocks a denylisted path, an oversized diff, anything shaped like a secret > grade - a second agent, fresh context, grades against rubrics it never wrote > receipt - writes receipt.json, appends the trace, updates the checkpoint three brakes stack in order: > daily spend cap, checked before the shift starts > per-call cap, checked before each model call > hard timeout for whatever's already running scheduling runs on schedule.yml and one cron line run-due is idempotent - a missed window is a skipped shift, never a backlog dumped at once The official ca: 0x642d30c84211ade7768fe557fbaed7224e2068c7 50 tests cover the parsers, the lock, the guard, the ledger under concurrent writers, and the kill switch
0:12

LOOP RAT: THE TECH UNDER AN AUTONOMOUS CODING LOOP no framework, no service, no database a dozen files of bash and python, plus one per loop one shift runs six phases: > preflight - won't start if state/HALT exists, spend is gone, or the loop's already running > act - runs under a hard timeout. shell gathers facts, the model only judges > verify - runs the plan's verify: command. "tests pass" isn't proof. the exit code is > guard - fingerprints the tree. blocks a denylisted path, an oversized diff, anything shaped like a secret > grade - a second agent, fresh context, grades against rubrics it never wrote > receipt - writes receipt.json, appends the trace, updates the checkpoint three brakes stack in order: > daily spend cap, checked before the shift starts > per-call cap, checked before each model call > hard timeout for whatever's already running scheduling runs on schedule.yml and one cron line run-due is idempotent - a missed window is a skipped shift, never a backlog dumped at once The official ca: 0x642d30c84211ade7768fe557fbaed7224e2068c7 50 tests cover the parsers, the lock, the guard, the ledger under concurrent writers, and the kill switch

Mr. Buzzoni

11,827 Aufrufe • vor 1 Monat

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.
0:18

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Akshay 🚀

245,663 Aufrufe • vor 3 Monaten

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example:
0:13

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example:

Sergio Paniego

38,088 Aufrufe • vor 2 Monaten

anthropic will sell you opus 5 at $200/mo. openai will sell you gpt-5.6 at $200/mo. neither will tell you the fix that drops your bill to $20 was posted free on langchain's blog on july 18 peter steinberger posted one line asking if we'd moved from loops to graphs yet. 24 hours later there was a manifesto. a week later every ai account had a $497 graph engineering course. all of them wrong about the same thing the sentence that ends the argument, buried in a langchain post nobody quoted: loop engineering isn't an alternative to graphs, so much as a simple version of them the machine, five layers, each wraps the one below: L1 the ask · 23% of errors (anthropic red team, q4 2024) -> "just add more instructions" burns tokens with zero accuracy gain -> real fix: examples, output schema, constraints as positives L2 the context · where 90% of you actually die -> 140,500 tokens where 18,000 would work, 8x the price for the worse answer -> real fix: retrieve, rank, compact, clear dead tool outputs L3 the harness · 31% of "model bugs" are harness bugs (openai safety eval, 2024) -> unbounded file perms = avg $23,400 incident. sandboxed = $0 (stripe internal) -> no timeout = $847 median in api fees before you notice -> real fix: explicit scopes, timeouts, human-required gates L4 the loop · "it stopped" is a loop exit problem -> the verifier said "looks good" to garbage. again -> real fix: machine-checkable exit test, turn cap, rubric L5 the graph · only 12% of teams use graphs in prod (stanford hai, n=2,841) -> 58% of graph failures are wrong-agent selection, not model -> teams abandon graphs saying "harder to debug than a loop." that's a harness problem -> real fix: name every node's specialty, delete decoration fix down, not up. a symptom at layer 4 usually originates at layer 2. a bigger model on a broken harness is a smarter employee locked in the same empty room drop your $200/mo ai sub to $20, check the article below
0:19

anthropic will sell you opus 5 at $200/mo. openai will sell you gpt-5.6 at $200/mo. neither will tell you the fix that drops your bill to $20 was posted free on langchain's blog on july 18 peter steinberger posted one line asking if we'd moved from loops to graphs yet. 24 hours later there was a manifesto. a week later every ai account had a $497 graph engineering course. all of them wrong about the same thing the sentence that ends the argument, buried in a langchain post nobody quoted: loop engineering isn't an alternative to graphs, so much as a simple version of them the machine, five layers, each wraps the one below: L1 the ask · 23% of errors (anthropic red team, q4 2024) -> "just add more instructions" burns tokens with zero accuracy gain -> real fix: examples, output schema, constraints as positives L2 the context · where 90% of you actually die -> 140,500 tokens where 18,000 would work, 8x the price for the worse answer -> real fix: retrieve, rank, compact, clear dead tool outputs L3 the harness · 31% of "model bugs" are harness bugs (openai safety eval, 2024) -> unbounded file perms = avg $23,400 incident. sandboxed = $0 (stripe internal) -> no timeout = $847 median in api fees before you notice -> real fix: explicit scopes, timeouts, human-required gates L4 the loop · "it stopped" is a loop exit problem -> the verifier said "looks good" to garbage. again -> real fix: machine-checkable exit test, turn cap, rubric L5 the graph · only 12% of teams use graphs in prod (stanford hai, n=2,841) -> 58% of graph failures are wrong-agent selection, not model -> teams abandon graphs saying "harder to debug than a loop." that's a harness problem -> real fix: name every node's specialty, delete decoration fix down, not up. a symptom at layer 4 usually originates at layer 2. a bigger model on a broken harness is a smarter employee locked in the same empty room drop your $200/mo ai sub to $20, check the article below

starmex

147,282 Aufrufe • vor 2 Monaten

Let me explain the agent loop, simple It's the core of every agentic system, and the part most people overcomplicate It's just this: 1. Send messages to the model 2. Model responds, maybe calls a tool 3. You run the tool 4. Append the result back to messages 5. Repeat until stop_reason is end_turn Step 4 is the whole thing, the write-back is what makes it an agent The model has to see what actually happened before it decides the next move That's the entire loop... understand this cold before you reach for a framework
0:14

Let me explain the agent loop, simple It's the core of every agentic system, and the part most people overcomplicate It's just this: 1. Send messages to the model 2. Model responds, maybe calls a tool 3. You run the tool 4. Append the result back to messages 5. Repeat until stop_reason is end_turn Step 4 is the whole thing, the write-back is what makes it an agent The model has to see what actually happened before it decides the next move That's the entire loop... understand this cold before you reach for a framework

Daniel San

12,514 Aufrufe • vor 4 Monaten

We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.
0:13

We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.

NVIDIA AI

763,100 Aufrufe • vor 3 Monaten

Claude can't, but GPT 5.6 on GOD-MODE is IMMACULATE Here's how you create it step by step: > open the new desktop app, pick Sol, reasoning on High. this is taste work, don't give it to the small models > feed it 2-3 sites you love and one line: "extract the art direction: mood, motion, typography, pacing. write it down as a style bible" > then the brief, one paragraph, goal not steps: "resort site for [name]. cinematic scroll, the booking button always one glance away. follow the style bible" > add house rules: no template hero-with-three-cards, no stock gradients, motion carries the story, every section earns its scroll > set the bar: "a working designer can't tell this from an agency build." then spin up a SECOND 5.6 with fresh context to grade against that bar. the builder never grades itself > loop it: build, grade, close the biggest gap, again. walk away, it doesn't need you in the room > when the verifier runs out of complaints: tag Sites. live URL, one click, no hosting, no deploy The deeper version of every step (the contract, the rules, the verifier trick, when to spend on Ultra) is in the article below.
0:10

Claude can't, but GPT 5.6 on GOD-MODE is IMMACULATE Here's how you create it step by step: > open the new desktop app, pick Sol, reasoning on High. this is taste work, don't give it to the small models > feed it 2-3 sites you love and one line: "extract the art direction: mood, motion, typography, pacing. write it down as a style bible" > then the brief, one paragraph, goal not steps: "resort site for [name]. cinematic scroll, the booking button always one glance away. follow the style bible" > add house rules: no template hero-with-three-cards, no stock gradients, motion carries the story, every section earns its scroll > set the bar: "a working designer can't tell this from an agency build." then spin up a SECOND 5.6 with fresh context to grade against that bar. the builder never grades itself > loop it: build, grade, close the biggest gap, again. walk away, it doesn't need you in the room > when the verifier runs out of complaints: tag Sites. live URL, one click, no hosting, no deploy The deeper version of every step (the contract, the rules, the verifier trick, when to spend on Ultra) is in the article below.

Miraqle

138,401 Aufrufe • vor 3 Monaten

I STOPPED REVIEWING MY OWN AGENT, SOMETHING ELSE DOES IT NOW I used to read every diff it produced and approve most of them, because an agent grading itself always says the work is good. -> Now a second model with different instructions tries to break the work first, and I only read what survived. Here is what is actually in the folder that took over the night shift: • the brief > CONTRACT.md -- what it may touch, and what it may never touch. > VISION.md -- the destination, so turn 47 still knows why it started. • the gate > judge/ -- a different model, never the one that wrote the code. > break-it.md -- it opens the page, clicks, screenshots, reports back. > -- no opinion, just zero or non-zero. > shift.yml -- 03:30 every night, laptop closed. • the memory > receipts/ -- one folder per night, dated and graded. > STATE.md -- where it stopped and what it escalated. > lessons.log -- the flaky test, written down once instead of rediscovered weekly. • the brakes > caps.json -- turn limit, retry limit, spend limit. > -- written on day one, used never. The generator decides what your loop can produce -> The judge decides what it refuses to produce. One of those is the part everyone builds -> The other is why most loops quietly fail. Bookmark it & Read Full breakdown below ↓
0:18

I STOPPED REVIEWING MY OWN AGENT, SOMETHING ELSE DOES IT NOW I used to read every diff it produced and approve most of them, because an agent grading itself always says the work is good. -> Now a second model with different instructions tries to break the work first, and I only read what survived. Here is what is actually in the folder that took over the night shift: • the brief > CONTRACT.md -- what it may touch, and what it may never touch. > VISION.md -- the destination, so turn 47 still knows why it started. • the gate > judge/ -- a different model, never the one that wrote the code. > break-it.md -- it opens the page, clicks, screenshots, reports back. > -- no opinion, just zero or non-zero. > shift.yml -- 03:30 every night, laptop closed. • the memory > receipts/ -- one folder per night, dated and graded. > STATE.md -- where it stopped and what it escalated. > lessons.log -- the flaky test, written down once instead of rediscovered weekly. • the brakes > caps.json -- turn limit, retry limit, spend limit. > -- written on day one, used never. The generator decides what your loop can produce -> The judge decides what it refuses to produce. One of those is the part everyone builds -> The other is why most loops quietly fail. Bookmark it & Read Full breakdown below ↓

slash1s

37,556 Aufrufe • vor 2 Monaten