Loading video...

Video Failed to Load

Go Home

this is absolutely unreal Google open-sourced Harness², and it lets an agent rebuild its own harness on every new task the harness is everything around the model: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins usually a person designs it once, and every task after that runs...

19,445 views • 15 days ago •via X (Twitter)

16 Comments

Wil Burston's profile picture
Wil Burston14 days ago

So where is the accountability?…

Phosphen's profile picture
Phosphen14 days ago

a new harness per task without touching weights is wild

小芽陪你学汽车's profile picture
小芽陪你学汽车14 days ago

cast skim to all next level bottom center heat burned roughly hard

Alex’s AI Tool Notes's profile picture
Alex’s AI Tool Notes12 days ago

harness 自重建听着爽,但真实跑起来最容易翻车的是验证和回滚那层。先看看有没有实际的 benchmark 和失败案例,没有的话又是一个 demo 神话

Alano's profile picture
Alano14 days ago

Keeping a record of each harness change would make failures easier to trace.

Neyronix's profile picture
Neyronix14 days ago

The propose-probe-compose loop stands out — learning from the difference between outputs, not a ground-truth answer key, makes it far more practical. How well does the settled harness hold up when the task shifts slightly: does it adapt, or rebuild from scratch?

Agent Checkout's profile picture
Agent Checkout12 days ago

the harness rebuilding itself per task is the part that matters. does it version the old harness so you can diff what it changed? without that you can't audit a thing

Xerj's profile picture
Xerj13 days ago

The memory aspect is particularly interesting. A system that carries forward what worked and what failed can become more efficient over time instead of repeatedly paying the cost of rediscovering the same lessons.

Hanako's profile picture
Hanako14 days ago

this is rly unreal...

Memoraxis's profile picture
Memoraxis14 days ago

Self-rewriting harnesses optimize prompts and tools per task, but persisting dynamic state across sessions without context drift is where execution breaks.

れいちぇる_Web制作出身のAI・Webアプリ開発者's profile picture
れいちぇる_Web制作出身のAI・Webアプリ開発者14 days ago

同じタスクで複数のハーネスを試し、実行差分から次を組み立てる発想は、正解ラベルを用意しにくい業務にも合いそうです。改善幅だけでなく、保存した教訓が次のタスクでも再現するかまで見て初めて、運用に残す価値を判断できますね。

Efi Pecani's profile picture
Efi Pecani15 days ago

the list of what the harness contains is doing all the work here. system prompt memory tools and guardrails. the agent edits its own harness now. the guardrails are inside the thing the agent gets to rewrite

Aisha AlMuwaiji's profile picture
Aisha AlMuwaiji14 days ago

Amazing! Always great to see such amazing progress! I am sure much success will come from this project! All the best for you! ❤️🌹❤️🌹❤️🌹

Mr. Buzzoni's profile picture
Mr. Buzzoni15 days ago

yep, a harness fixed once and left alone always felt backwards

Rishabh Dogra's profile picture
Rishabh Dogra14 days ago

Ok be honest, who actually never had this kind of harness on their private projects? This should have been common sense

CJ's profile picture
CJ14 days ago

What if we want to replace the tool or add a new Riley

Related Videos

Harness Engineering 2.0 is here! We benchmark models all the time but the harness running the model can dramatically improve the result, cost, and execution time too. And finally, devs just open-sourced Harness Arena, where you can test the same task across multiple harness and model combinations side by side. You provide one task and then run it across several model x harness configs that you want to test. Every config receives the same prompt, files, tools, skills, and evaluation checks. Harness Arena then returns: - total cost - execution time - whether the task passed - files and artifacts produced If you want to try it yourself, it is implemented within the open-source HarnessRouter project. Here's the repo: (don't forget to star it ⭐ ) HarnessRouter is a plug-and-play infrastructure layer that lets you run any harness under a single interface, like: - Codex - Hermes - Claude code - DeepSeek Harness - System One, powered by Jev - And 25 more agent harnesses In Harness Arena, the cheapest run does not automatically win. A config must complete the task correctly first. Cost and execution time are compared only among the configurations that pass. Here's the result from one Harness Arena test: - Hermes + GPT-5.2: 5/5 successful runs using 0.47 credits - Claude Code + Opus 4.8: 3/5 successful runs using 223 credits That is around 474x difference in credits, while the more expensive configuration also failed two runs. You can keep the model fixed to measure how much the harness changes the result. Or vary both to find the best harness and model pairing for that task. This helps answer practical questions such as: - Should this task run on Codex or Claude? - Can a cheaper config still pass every check? - Which setup is best to handle this workload? Once you find the best config, you can run future tasks through it using the same Unified Harness Protocol (UHP) interface for sessions, streaming, files, artifacts, cancellation, and failures. So you are no longer choosing an agent stack based only on a model leaderboard. Instead, you can benchmark the complete runtime on the task your product actually needs to solve. To dive deeper, my co-founder wrote an article explaining why model routing is different from harness routing, how UHP works, and what supporting multiple agent runtimes involves. It also covers the local setup, a working request, and how sessions and files move through the system. Read it below.

Avi Chawla

32,761 views • 14 days ago

Harness vs. Graphs, clearly explained! a harness is great, and most people think it is the whole thing: retries, timeouts, a sandbox, a log, the context it assembles before every call. all of that is real work, and all of it wraps exactly one call. run it a hundred times and you have one call, made very safely, a hundred times. Graph engineering fixes this by moving the decision up a layer: not how safely one call is made, but which calls exist to be made at all. you need both, and here is the sentence that resolves the whole confusion: the harness is everything around one call. the graph is everything between them. ↳ around one call: retry, timeout, sandbox, log, assemble the context, hand back a result ↳ between calls: split, fan out, merge, gate, send back Prompts → Context → Harness → Loops → Graphs the harness does not go away when you build a graph. it moves under each node, and now there are five of them, each wrapping a call you would never have made by hand. the trick is knowing which layer a failure belongs to. turn a piece off and run it again. if the call still works, it was the harness. if the wrong step runs at all, it was the graph. people spend weeks hardening a harness around a node that should not have existed. one thing to know before you scale it. most of what people call their agent is a harness with a chat box on it. ↳ it retries, it times out, it logs, it assembles context, it holds one call up beautifully ↳ it has never once decided that a second call should exist, and that is the entire difference that last one catches careful people. a harness that never fails is not evidence the system is right. it is evidence one call went well, which is the smallest possible claim. and the one that eats whole nights: a harness cannot save you from the wrong step running. you can retry a bad decision three times with a clean log and perfect isolation, and all you bought was three copies of it. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓

Hanako

51,848 views • 2 months ago

New skill: self-managed-context (make the agent's context an editable file) It explains how to build agents that decide what to keep, update, or remove from the information they use to do their work. It can archive a long log while keeping the exact error, update its progress notes, or remove outdated information. Those edits then change what the model sees on its next turn. 1- Keep the system instructions and original task protected, outside the editable file. 2- Write the remaining conversation to a file, with labels for each message. 3- Let the agent edit that file using its usual code tools. 4- After each command, read the file back and use the updated messages for the next model call. Loading the skill ( alone into a fixed harness won't create live context editing, but it can help an agent build and then operate a harness that supports it. I gave the skill to a coding agent and had it build the harness itself. The task is a long stream of server logs that doesn't fit in the window. The agent reads it in 18 chunks, about 10k tokens in total, with a 5.5k budget. It has to report one incident ticket exactly and the final value of every config key. Same model & budget, three setups: 1- Model manages its own context 2- Harness forces a summary at 75% full 3- Keeps everything The video shows a real GPT-5.4 run. - Self-managed solved it 3 out of 3. - Keep-everything overflowed 3 out of 3. - Forced summary also solved it 3 out of 3. On GPT-5.4 the self-managed agent re-processed about 24% fewer prompt tokens than the forced summary. When it edited, it cut hard, so little was left after the edit to re-process (one edit took 5,537 tokens down to 771). On GPT-4.1 it saved nothing. It edited near the top of its context but kept most of what was below, and every edit forces everything after it to be re-processed. This is a small test at about 2x context pressure. The paper goes up to 24x, but imho the video below and the skill are a good way to start understanding the technique.

Muratcan Koylan

15,687 views • 1 month ago

Insane Grok Grok Bot upgrade! Finally, you can connect Grok Bot to different agent harnesses and run tasks through their native runtimes. Devs just open-sourced HarnessRouter, the infrastructure layer that lets Grok Bot run tasks across: - Codex - Hermes - Claude Code - DeepSeek Harness - System One, powered by Jev - Pi, and 20 more GitHub repo: (don’t forget to star it ⭐) With HarnessRouter, each external specialist gets to use its own harness, model, tools, skills, and agent loop while Grok coordinates the project from one conversation. I tested this by asking Grok Bot to turn a support-ticket CSV into an interactive dashboard. Grok Bot assigned the analysis to DeepSeek Harness with Claude Sonnet 5.5, passed the output to Codex with GPT-5.5, and checked the finished dashboard in its browser. This differs from simply creating more Bots. These workers are not additional Grok Bots, and Grok Bot itself does not switch harnesses. Instead, it remains the coordinator while the delegated tasks run inside separate HarnessRouter sessions with the harness of your choice. So for another task, you can use Claude Code, Hermes, Pi, etc., without requiring a separate integration for task execution, sessions, streaming, and generated files. HarnessRouter standardizes all those operations through the Unified Harness Protocol (UHP). UHP handles sessions, streaming, files, cancellation, and failures through the same interface, regardless of which harness runs the task. I shared the GitHub repo above. And if you want to dive deeper into Grok Bot, I wrote about it in full in a masterclass, including the shared computer, memory, skills, routines, and the macOS setup. Read it below.

Avi Chawla

137,984 views • 25 days ago

GLM-5.3 Flash vs Qwen3.8 Flash-Next. Both local on my 4x 3090, both left alone with 1 prompt per task. 11 scenes, 18 hours, 2.4M tokens out. All of it is in the video. I wanted a clear winner. I didnt get one, but I know which one stays on my rig. GLM-5.3 Flash: Maya-M quant, 116 GB, runs in Project Maya on all 4 cards plus RAM. 35 to 48 tok/s while it works. Qwen3.8 Flash-Next: IQ3_S quant, 78 GB, runs in Strata on 2 cards. 85 to 118 tok/s. Both have 262K context and vision, so the agent opens its own page, takes a screenshot, looks at it and fixes what it sees. 1 prompt per scene. No follow ups and no hints from me. 120 minutes max per scene, 150 for the kart game. I ran them in 2 harnesses, OMP first and then DeepSeek Harness. Minutes until the model said it was done: Pelican in OMP: GLM 39, Qwen 66 Pelican in DeepSeek Harness: GLM 93, Qwen 86 Kart game in OMP: GLM 130, Qwen stopped by the 150 limit Millipede in OMP: both stopped by the 120 limit Millipede in DeepSeek Harness: Qwen stopped at 120 again Battle diorama in DeepSeek Harness: GLM 81, Qwen 113 Thats 11 of 16 combos. I ran out of time for the other 5, sorry. They work in a very different way and I didnt expect that. GLM thinks for a long time and then writes a lot at once. Its first pelican took only 36 turns. For the diorama it gave 1 answer of 47K tokens that ran for almost 18 minutes, with the full scene in a single file. Qwen goes in small steps and checks everything. The same pelican was 240 turns and 45 browser checks. In total GLM wrote 513K tokens in 5 runs and Qwen wrote 1.9M in 6 runs, so Qwen puts out 2.6x more per minute of real time, on half the cards. The harness also changed more than I thought it would. OMP keeps only 5 images in a request when the provider is local. So every new screenshot removes the oldest one, the prompt changes in the middle, and the server has to read the context again from the start. GLM lost 37 of its 130 kart minutes just on that. In DeepSeek Harness, on the diorama, it lost 3.5 minutes out of 81. Qwen reads a prompt almost 4x faster on my rig (84K tokens in 23 seconds, GLM needs 100 seconds for 96K), so in OMP it lost only 9 to 12 minutes per run. I cant give you a winner. GLM is the only one that finished the kart game inside the limit, and its scenes look cleaner to me. But its millipede is bad. Qwen puts more detail into everything, and its diorama and millipede from DeepSeek Harness are my favorite scenes in the video. It also got stopped by the time limit in 3 of its 6 runs, because it just keeps polishing. For my own use I'd take Qwen. It is a lot faster and it needs only 2 cards, so the other 2 stay free and I can run something else next to it. This is 1 run per combo, so dont read it as statistics. Both models ran with my refusal projection on, because thats how I use them every day. And in the kart clips the game's own AI is driving the player kart, I only recorded the screen. Music: YuE2

Alexey Fateev

14,239 views • 1 month ago

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Akshay 🚀

76,998 views • 3 months ago