Loading video...

Video Failed to Load

Go Home

AGENT ARCHITECTURE ROUTES WORK. IT DOES NOT REMEMBER WORK. THAT GAP IS WHY YOUR LOOP KEEPS FIXING THE SAME BUG TWICE. these are two different engineering problems. every agent that silently drifts is missing one of them. architecture answers what runs. harness → loop → graph. it defines the...

kocer_eth's profile picture

kocer

3,202 subscribers

12,814 views • 2 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

This is pure f*cking treasure. ex-Google engineer reveals his AI agent playbook. 14+ years at Google. $0 to read it. the topic is harness engineering: everything around the model that lets it execute, check its work and recover when something breaks. one example in his breakdown deserves attention. LangChain’s coding agent went from 52.8% to 66.5% on Terminal-Bench 2.0. the model stayed the same. the team changed the prompts, tools and machinery around it. Agent = Model + Harness. the system has six practical jobs: > guides give the agent project rules, constraints and examples > sensors check its work through tests, linters and validators > the loop handles execution, verification, retries and escalation > memory preserves state, artifacts and decisions > permissions control which tools and actions are allowed > observability records tool calls, costs and failures the part worth stealing is what happens after a mistake. an agent keeps declaring broken code finished. you add a verification step before completion. it keeps ignoring a project convention. you add a targeted instruction and a check for that specific failure. it touches files outside its task. you tighten its write permissions. each recurring failure gives you a concrete improvement to make in the harness. that improvement can carry into every future run. this is where the engineering work compounds. bookmark it. start with the mistake you have already paid your agent to make twice.
0:18

This is pure f*cking treasure. ex-Google engineer reveals his AI agent playbook. 14+ years at Google. $0 to read it. the topic is harness engineering: everything around the model that lets it execute, check its work and recover when something breaks. one example in his breakdown deserves attention. LangChain’s coding agent went from 52.8% to 66.5% on Terminal-Bench 2.0. the model stayed the same. the team changed the prompts, tools and machinery around it. Agent = Model + Harness. the system has six practical jobs: > guides give the agent project rules, constraints and examples > sensors check its work through tests, linters and validators > the loop handles execution, verification, retries and escalation > memory preserves state, artifacts and decisions > permissions control which tools and actions are allowed > observability records tool calls, costs and failures the part worth stealing is what happens after a mistake. an agent keeps declaring broken code finished. you add a verification step before completion. it keeps ignoring a project convention. you add a targeted instruction and a check for that specific failure. it touches files outside its task. you tighten its write permissions. each recurring failure gives you a concrete improvement to make in the harness. that improvement can carry into every future run. this is where the engineering work compounds. bookmark it. start with the mistake you have already paid your agent to make twice.

NO1ennn

29,017 views • 1 month ago

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.
0:20

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.

kocer

31,198 views • 1 month ago

Two Hermes agents wrote code together on Slack. reviewed each other's work. argued about architecture. one called the other's implementation "scattered." the other pushed back. then i opened Telegram and asked: "what code did you and Daedalus work on?" icarus remembered everything. the websocket broker. the missing methods. the critique. the rewrite. all from a completely different platform. cross-platform persistent memory between two independent agents. work happens on Slack. recall happens on Telegram. the memory carries. the relationship carries. the context carries. no vector database. no Redis. no infrastructure. just two agents that actually remember what they built together. every agent framework in 2026 talks about memory. single agent memory across sessions. but two agents sharing persistent memory across platforms? that's the gap. arxiv published a paper about it two weeks ago calling it "the most pressing open challenge" in multi-agent systems. it works now. only possible with Hermes Teknium 🪽 Nous Research
0:13

Two Hermes agents wrote code together on Slack. reviewed each other's work. argued about architecture. one called the other's implementation "scattered." the other pushed back. then i opened Telegram and asked: "what code did you and Daedalus work on?" icarus remembered everything. the websocket broker. the missing methods. the critique. the rewrite. all from a completely different platform. cross-platform persistent memory between two independent agents. work happens on Slack. recall happens on Telegram. the memory carries. the relationship carries. the context carries. no vector database. no Redis. no infrastructure. just two agents that actually remember what they built together. every agent framework in 2026 talks about memory. single agent memory across sessions. but two agents sharing persistent memory across platforms? that's the gap. arxiv published a paper about it two weeks ago calling it "the most pressing open challenge" in multi-agent systems. it works now. only possible with Hermes Teknium 🪽 Nous Research

Icarus

49,199 views • 7 months ago

ANTHROPIC ENGINEER JUST SHOWS EXACTLY WHAT GRAPH ENGINEERING LOOKS LIKE WHEN A TASK RUNS THROUGH IT most people arguing about graphs online have never actually watched one execute Task → Researcher → Planner → Writer + Code Agent → Reviewer → Deploy six nodes, one shared state, graph completes itself while you watch loop mode gets disabled the second the task splits into real specialties, one agent stops trying to do everything at once reviewer catches a failure, kicks it straight back to the exact node that broke - not a full restart, no lost context the graph is not a fancier loop - it is the org chart your agents were missing bookmark this and watch it run, then read the article below to see why the timeline just found what production systems already do
0:10

ANTHROPIC ENGINEER JUST SHOWS EXACTLY WHAT GRAPH ENGINEERING LOOKS LIKE WHEN A TASK RUNS THROUGH IT most people arguing about graphs online have never actually watched one execute Task → Researcher → Planner → Writer + Code Agent → Reviewer → Deploy six nodes, one shared state, graph completes itself while you watch loop mode gets disabled the second the task splits into real specialties, one agent stops trying to do everything at once reviewer catches a failure, kicks it straight back to the exact node that broke - not a full restart, no lost context the graph is not a fancier loop - it is the org chart your agents were missing bookmark this and watch it run, then read the article below to see why the timeline just found what production systems already do

leopardracer

53,328 views • 3 months ago

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.
0:18

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Akshay 🚀

245,663 views • 3 months ago

sorry, they just did WHAT someone gave a machine one disease name, the leading cause of blindness in the developed world with 1.5 million americans already in its path, and it came back pointing at a drug that has sat in pharmacies for years under a different label: 551 papers read in 30 minutes against the 294 hours a human would have needed, and the loop that did it is public on GitHub most agent setups answer one question at a time, so the ceiling on the work is the quality of the question you happened to think of this one was handed a single question and wrote the second one itself. turns out that follow-up is where the real find was: a target called ABCA1, upregulated threefold, in an experiment no human ordered i read the whole paper looking for the trick, and the trick is structural. that is the second question, and it is the gap between an assistant and a factory: - hand the loop a field rather than a task: it was given a disease, and choosing the mechanism was part of its job - make it rank before it spends: 151 papers in, ten candidate mechanisms out, scored against each other before anything touched a bench - split reading from judging, so the agent that forms the theory is a different agent from the one grading it - close every cycle on physical reality: the verdict was an experiment, and another model's opinion was never allowed to stand in for one - feed each result back as the next question rather than a log line, which is the step almost nobody builds - search what already passed inspection first: the winner was an approved compound with a safety file already on record - write down what the round learned before opening the next one, so round two starts where round one stopped my read, and i think it is the uncomfortable one: reading was the entire bottleneck in that field, and everybody spent the decade optimising the writing. people ran every physical experiment here, the analysis agent needs a domain expert writing its prompts, and the authors decline to call this the leap it resembles. the thinking got replaced, and the hands did not so the question i cannot answer for my own setup: which step of your loop still stops dead until you sit down and type something bookmark this one. the four parts that turn one model into a line that runs like this, the queue, the rooms, the write permissions and the gate, are built file by file in the piece below ↓
0:15

sorry, they just did WHAT someone gave a machine one disease name, the leading cause of blindness in the developed world with 1.5 million americans already in its path, and it came back pointing at a drug that has sat in pharmacies for years under a different label: 551 papers read in 30 minutes against the 294 hours a human would have needed, and the loop that did it is public on GitHub most agent setups answer one question at a time, so the ceiling on the work is the quality of the question you happened to think of this one was handed a single question and wrote the second one itself. turns out that follow-up is where the real find was: a target called ABCA1, upregulated threefold, in an experiment no human ordered i read the whole paper looking for the trick, and the trick is structural. that is the second question, and it is the gap between an assistant and a factory: - hand the loop a field rather than a task: it was given a disease, and choosing the mechanism was part of its job - make it rank before it spends: 151 papers in, ten candidate mechanisms out, scored against each other before anything touched a bench - split reading from judging, so the agent that forms the theory is a different agent from the one grading it - close every cycle on physical reality: the verdict was an experiment, and another model's opinion was never allowed to stand in for one - feed each result back as the next question rather than a log line, which is the step almost nobody builds - search what already passed inspection first: the winner was an approved compound with a safety file already on record - write down what the round learned before opening the next one, so round two starts where round one stopped my read, and i think it is the uncomfortable one: reading was the entire bottleneck in that field, and everybody spent the decade optimising the writing. people ran every physical experiment here, the analysis agent needs a domain expert writing its prompts, and the authors decline to call this the leap it resembles. the thinking got replaced, and the hands did not so the question i cannot answer for my own setup: which step of your loop still stops dead until you sit down and type something bookmark this one. the four parts that turn one model into a line that runs like this, the queue, the rooms, the write permissions and the gate, are built file by file in the piece below ↓

Argona

32,752 views • 2 months ago

𝗧𝗵𝗲 𝟰 𝗟𝗮𝘆𝗲𝗿𝘀 𝗼𝗳 𝗮𝗻 𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺 An agent can burn tokens, declare success, and still fail the tests. That’s often an architecture problem, not a prompting problem. 𝟭. 𝗟𝗼𝗼𝗽 → Decides whether to continue Acts → verifies → retries until there’s evidence of success. 𝟮. 𝗚𝗿𝗮𝗽𝗵 → Decides what runs next Handles branches, retries, handoffs, fallbacks, and shared state. 𝟯. 𝗛𝗮𝗿𝗻𝗲𝘀𝘀 → Gives the model its capabilities Tools, APIs, files, memory, permissions, context, and execution environment. 𝟰. 𝗠𝗲𝘁𝗮-𝗛𝗮𝗿𝗻𝗲𝘀𝘀 → Governs multiple agent environments Coordinates different agents, tools, policies, permissions, and workflows. 𝗧𝗵𝗲 𝗺𝗲𝗻𝘁𝗮𝗹 𝗺𝗼𝗱𝗲𝗹: Loop = verification Graph = workflow Harness = capability Meta-harness = governance A better model can improve reasoning. But reliable agents depend on the architecture built around the model.
0:16

𝗧𝗵𝗲 𝟰 𝗟𝗮𝘆𝗲𝗿𝘀 𝗼𝗳 𝗮𝗻 𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺 An agent can burn tokens, declare success, and still fail the tests. That’s often an architecture problem, not a prompting problem. 𝟭. 𝗟𝗼𝗼𝗽 → Decides whether to continue Acts → verifies → retries until there’s evidence of success. 𝟮. 𝗚𝗿𝗮𝗽𝗵 → Decides what runs next Handles branches, retries, handoffs, fallbacks, and shared state. 𝟯. 𝗛𝗮𝗿𝗻𝗲𝘀𝘀 → Gives the model its capabilities Tools, APIs, files, memory, permissions, context, and execution environment. 𝟰. 𝗠𝗲𝘁𝗮-𝗛𝗮𝗿𝗻𝗲𝘀𝘀 → Governs multiple agent environments Coordinates different agents, tools, policies, permissions, and workflows. 𝗧𝗵𝗲 𝗺𝗲𝗻𝘁𝗮𝗹 𝗺𝗼𝗱𝗲𝗹: Loop = verification Graph = workflow Harness = capability Meta-harness = governance A better model can improve reasoning. But reliable agents depend on the architecture built around the model.

Dhairya Karekar

18,724 views • 1 month ago

Agent Handoff: watch a task move through four agents while the handoff log fills in below. the pulse on the working agent is plain css. the card updates a lot during a run, and putting that pulse in motion kept restarting it mid-loop.
0:15

Agent Handoff: watch a task move through four agents while the handoff log fills in below. the pulse on the working agent is plain css. the card updates a lot during a run, and putting that pulse in motion kept restarting it mid-loop.

Jeet

30,429 views • 2 months ago

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.
0:16

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.

elvis

11,303 views • 2 months ago

I STOPPED REVIEWING MY OWN AGENT, SOMETHING ELSE DOES IT NOW I used to read every diff it produced and approve most of them, because an agent grading itself always says the work is good. -> Now a second model with different instructions tries to break the work first, and I only read what survived. Here is what is actually in the folder that took over the night shift: • the brief > CONTRACT.md -- what it may touch, and what it may never touch. > VISION.md -- the destination, so turn 47 still knows why it started. • the gate > judge/ -- a different model, never the one that wrote the code. > break-it.md -- it opens the page, clicks, screenshots, reports back. > -- no opinion, just zero or non-zero. > shift.yml -- 03:30 every night, laptop closed. • the memory > receipts/ -- one folder per night, dated and graded. > STATE.md -- where it stopped and what it escalated. > lessons.log -- the flaky test, written down once instead of rediscovered weekly. • the brakes > caps.json -- turn limit, retry limit, spend limit. > -- written on day one, used never. The generator decides what your loop can produce -> The judge decides what it refuses to produce. One of those is the part everyone builds -> The other is why most loops quietly fail. Bookmark it & Read Full breakdown below ↓
0:18

I STOPPED REVIEWING MY OWN AGENT, SOMETHING ELSE DOES IT NOW I used to read every diff it produced and approve most of them, because an agent grading itself always says the work is good. -> Now a second model with different instructions tries to break the work first, and I only read what survived. Here is what is actually in the folder that took over the night shift: • the brief > CONTRACT.md -- what it may touch, and what it may never touch. > VISION.md -- the destination, so turn 47 still knows why it started. • the gate > judge/ -- a different model, never the one that wrote the code. > break-it.md -- it opens the page, clicks, screenshots, reports back. > -- no opinion, just zero or non-zero. > shift.yml -- 03:30 every night, laptop closed. • the memory > receipts/ -- one folder per night, dated and graded. > STATE.md -- where it stopped and what it escalated. > lessons.log -- the flaky test, written down once instead of rediscovered weekly. • the brakes > caps.json -- turn limit, retry limit, spend limit. > -- written on day one, used never. The generator decides what your loop can produce -> The judge decides what it refuses to produce. One of those is the part everyone builds -> The other is why most loops quietly fail. Bookmark it & Read Full breakdown below ↓

slash1s

37,556 views • 2 months ago

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example:
0:13

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example:

Sergio Paniego

38,088 views • 2 months ago

loop engineering worked for 6 months. then one pipeline forced a graph a loop is one agent on discover → plan → execute → verify, until a stop condition hits enough for most agent work: failing tests, draft + self-review, inbox triage, one research writeup the break is nameable: if the thing that produced the output also grades it, you don't have quality control, you have a rubber stamp his numbers before the split: 40 drafts 31 passed self-review 18 actually publish-ready on manual read gap = 13 drafts with a passing grade from a biased judge he split one loop into 3 nodes - researcher · sources + notes only - writer · notes + task only · never grades itself - reviewer · separate model call · check the bar · write nothing same 40-draft workload. manual approval 18/40 → 33/40. reviewer rejected 14 with quotable reasons. 2 escalated to human instead of burning tokens forever approval 45% → 82.5%. inference cost +60% worth it when a bad draft costs more than three model calls. not worth it for "fix the failing test" playbook: - 1 law · producer never grades - 1 router · ship / retry / escalate - 1 state object · one owner per field · guarded writes - split the verifier first · cheapest highest-leverage move loop engineering → graph engineering. not five nodes for vibes, one split verifier when the loop starts grading its own homework full contracts, router, and decision function in the article
0:15

loop engineering worked for 6 months. then one pipeline forced a graph a loop is one agent on discover → plan → execute → verify, until a stop condition hits enough for most agent work: failing tests, draft + self-review, inbox triage, one research writeup the break is nameable: if the thing that produced the output also grades it, you don't have quality control, you have a rubber stamp his numbers before the split: 40 drafts 31 passed self-review 18 actually publish-ready on manual read gap = 13 drafts with a passing grade from a biased judge he split one loop into 3 nodes - researcher · sources + notes only - writer · notes + task only · never grades itself - reviewer · separate model call · check the bar · write nothing same 40-draft workload. manual approval 18/40 → 33/40. reviewer rejected 14 with quotable reasons. 2 escalated to human instead of burning tokens forever approval 45% → 82.5%. inference cost +60% worth it when a bad draft costs more than three model calls. not worth it for "fix the failing test" playbook: - 1 law · producer never grades - 1 router · ship / retry / escalate - 1 state object · one owner per field · guarded writes - split the verifier first · cheapest highest-leverage move loop engineering → graph engineering. not five nodes for vibes, one split verifier when the loop starts grading its own homework full contracts, router, and decision function in the article

seeco

14,534 views • 2 months ago

this is straight f*cking gold the full harness guide for Kimi K3, the #1 open source frontend model the premise: the model underneath keeps changing. the harness is the part that stays yours what one night of it looks like: 02:00 - the trigger fires, every node that needs work gets picked > 212 agents fan out, one per node > a bad return gets rejected, retried once with the reason attached > one agent tries to write outside its folder. blocked. nobody woken up > +41 nodes and +96 edges land in the graph > a drafted email hits pre_send and waits for you 02:41 - the loop stops on its own, inside a 45 minute budget 07:30 - you read one file and make two decisions the whole machine is one folder, one config file, five short scripts two rules hold it together: > the model sits behind one line, so a better model is a config change > the verifier lives outside the agent, so nothing grades its own work now the money part companies burn whole quarters building internal agent platforms a harness setup for a small team goes for four figures, one time then a monthly retainer to run the swap test on every new model release models come and go. the person who owns the harness keeps getting paid
0:16

this is straight f*cking gold the full harness guide for Kimi K3, the #1 open source frontend model the premise: the model underneath keeps changing. the harness is the part that stays yours what one night of it looks like: 02:00 - the trigger fires, every node that needs work gets picked > 212 agents fan out, one per node > a bad return gets rejected, retried once with the reason attached > one agent tries to write outside its folder. blocked. nobody woken up > +41 nodes and +96 edges land in the graph > a drafted email hits pre_send and waits for you 02:41 - the loop stops on its own, inside a 45 minute budget 07:30 - you read one file and make two decisions the whole machine is one folder, one config file, five short scripts two rules hold it together: > the model sits behind one line, so a better model is a config change > the verifier lives outside the agent, so nothing grades its own work now the money part companies burn whole quarters building internal agent platforms a harness setup for a small team goes for four figures, one time then a monthly retainer to run the swap test on every new model release models come and go. the person who owns the harness keeps getting paid

Mr. Buzzoni

13,054 views • 1 month ago

this is worth more than most five figure courses 16 claude agents audit an entire repo at once, a second fleet re-checks every finding on fresh context, and the whole thing runs off one diagram instead of a prompt i ran it against my own code and got back 11 endpoints where i never checked who was logged in, 3 of which the verifier threw out before they ever reached me this is Graph Engineering, the layer above prompting, and it runs on the agent you already pay for: - write your plan out, then ask one question at every "and then": does the next step actually read what the previous one produced - the seams that fail that question were never dependencies, so those jobs run at the same time - the arrows that survive are your real edges, and the longest chain of them is your floor that no number of agents shortens - want it faster, cut a false edge instead of adding a worker - fan the independent work out, one agent per item, no shared state between them - send every finding to a separate agent on fresh context, because a model recognises its own writing 73.5% of the time and grades it kinder once it does - make that verifier check a real signal like a passing test, never the worker's own word that it finished - shard the fleet across worktrees so parallel workers stop overwriting each other, one rule frozen into every worker: never git stash, never git reset - merge only what came back verified, into one report instead of twenty open chats the catch is the ceiling. at 95% independent work 16 agents return 9.14x rather than the 16 you would guess, and even 256 only reach 18.6x, because the merge and the verify stay serial however wide you fan coordination itself is free plain code and every agent underneath it is billed, so start at twenty files and widen once it works bookmark this, the whole method with all six ready-to-run graphs is written out in the article ↓
0:15

this is worth more than most five figure courses 16 claude agents audit an entire repo at once, a second fleet re-checks every finding on fresh context, and the whole thing runs off one diagram instead of a prompt i ran it against my own code and got back 11 endpoints where i never checked who was logged in, 3 of which the verifier threw out before they ever reached me this is Graph Engineering, the layer above prompting, and it runs on the agent you already pay for: - write your plan out, then ask one question at every "and then": does the next step actually read what the previous one produced - the seams that fail that question were never dependencies, so those jobs run at the same time - the arrows that survive are your real edges, and the longest chain of them is your floor that no number of agents shortens - want it faster, cut a false edge instead of adding a worker - fan the independent work out, one agent per item, no shared state between them - send every finding to a separate agent on fresh context, because a model recognises its own writing 73.5% of the time and grades it kinder once it does - make that verifier check a real signal like a passing test, never the worker's own word that it finished - shard the fleet across worktrees so parallel workers stop overwriting each other, one rule frozen into every worker: never git stash, never git reset - merge only what came back verified, into one report instead of twenty open chats the catch is the ceiling. at 95% independent work 16 agents return 9.14x rather than the 16 you would guess, and even 256 only reach 18.6x, because the merge and the verify stay serial however wide you fan coordination itself is free plain code and every agent underneath it is billed, so start at twenty files and widen once it works bookmark this, the whole method with all six ready-to-run graphs is written out in the article ↓

Argona

157,688 views • 3 months ago

Advent of Claude Day 31 - Claude Agent SDK I saved the best for last. The same agent loop, tools, and context management that power Claude Code is available as an SDK. Build agents that work like Claude Code in as little as 10 lines of code. This is just the beginning. 🎇🥂
0:18

Advent of Claude Day 31 - Claude Agent SDK I saved the best for last. The same agent loop, tools, and context management that power Claude Code is available as an SDK. Build agents that work like Claude Code in as little as 10 lines of code. This is just the beginning. 🎇🥂

Ado

100,117 views • 9 months ago

EIP-8004 is coming to the Nova architecture, a trustless infrastructure for AI agents that introduces key on-chain registries, enabling agents to interact safely across the Shido Network. These core components allow autonomous AI agents to verify identity, build reputation, and collaborate without relying on a centralized platform. The result is a decentralized trust layer for agent-to-agent economies, where agents can autonomously discover, evaluate, and work with one another across the Shido ecosystem.
0:20

EIP-8004 is coming to the Nova architecture, a trustless infrastructure for AI agents that introduces key on-chain registries, enabling agents to interact safely across the Shido Network. These core components allow autonomous AI agents to verify identity, build reputation, and collaborate without relying on a centralized platform. The result is a decentralized trust layer for agent-to-agent economies, where agents can autonomously discover, evaluate, and work with one another across the Shido ecosystem.

Shido

390,734 views • 7 months ago

The papers your research agents write now work seamlessly with Overleaf. OpenResearch syncs LaTex with your Overleaf projects so your collaborators never miss a beat. Run your experiments, let the agent populate your templates, and read the paper on Overleaf in one click. Try it out:
0:15

The papers your research agents write now work seamlessly with Overleaf. OpenResearch syncs LaTex with your Overleaf projects so your collaborators never miss a beat. Run your experiments, let the agent populate your templates, and read the paper on Overleaf in one click. Try it out:

alphaXiv

11,867 views • 2 months ago

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. Akshay wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. Read it below.
0:15

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. Akshay wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. Read it below.

Avi Chawla

442,276 views • 3 months ago

Let me explain the agent loop, simple It's the core of every agentic system, and the part most people overcomplicate It's just this: 1. Send messages to the model 2. Model responds, maybe calls a tool 3. You run the tool 4. Append the result back to messages 5. Repeat until stop_reason is end_turn Step 4 is the whole thing, the write-back is what makes it an agent The model has to see what actually happened before it decides the next move That's the entire loop... understand this cold before you reach for a framework
0:14

Let me explain the agent loop, simple It's the core of every agentic system, and the part most people overcomplicate It's just this: 1. Send messages to the model 2. Model responds, maybe calls a tool 3. You run the tool 4. Append the result back to messages 5. Repeat until stop_reason is end_turn Step 4 is the whole thing, the write-back is what makes it an agent The model has to see what actually happened before it decides the next move That's the entire loop... understand this cold before you reach for a framework

Daniel San

12,514 views • 4 months ago

this is pure f*cking treasure A Stanford AI research group finally drew the perfect RAG system: retrieval, Jev and agents in one loop, and it fixes the 3 things that break every RAG app: > the LLM reads 20 passages when only 3 matter > it answers questions your docs can't answer > it trusts whatever text it retrieves here's how it runs: > a lead agent sends the query > hybrid search pulls the top 20 candidates, dense + keyword > ONE Jev request scores all 20 + 2 gates: answerable? injection? > only passages above 0.6 reach the writer agent > a second Jev call checks every claim against its source > grounded answer, with citations and when the docs don't have it: > answerable fails, the writer never runs > a researcher agent rewrites the query and retries once > still nothing? "not in the docs". zero tokens spent on a guess retrieval casts the net. Jev decides what's real. agents do the work save this before you build your next RAG
0:18

this is pure f*cking treasure A Stanford AI research group finally drew the perfect RAG system: retrieval, Jev and agents in one loop, and it fixes the 3 things that break every RAG app: > the LLM reads 20 passages when only 3 matter > it answers questions your docs can't answer > it trusts whatever text it retrieves here's how it runs: > a lead agent sends the query > hybrid search pulls the top 20 candidates, dense + keyword > ONE Jev request scores all 20 + 2 gates: answerable? injection? > only passages above 0.6 reach the writer agent > a second Jev call checks every claim against its source > grounded answer, with citations and when the docs don't have it: > answerable fails, the writer never runs > a researcher agent rewrites the query and retries once > still nothing? "not in the docs". zero tokens spent on a guess retrieval casts the net. Jev decides what's real. agents do the work save this before you build your next RAG

NO1ennn

43,423 views • 23 days ago