Loading video...

Video Failed to Load

Go Home

Google's team published a 9-page PDF on Harness Engineering. one formula replaced prompt engineering: Agent = Model + Harness the twist: same Claude Sonnet, same benchmark. change only the harness the playbook, six steps: step 1 -> add guides AGENTS.md, rule files, constraint docs. every line is a past...

79,906 views • 2 months ago •via X (Twitter)

22 Comments

AI Apps API's profile picture
AI Apps API2 months ago

Step four is the one people underrate and by far the hardest to retrofit. Externalizing memory is really three decisions: what gets extracted at write time, where it is stored, and how it comes back. Most teams make the second one and skip the first, dropping raw transcripts into a vector store, and then retrieval has to re-derive the entities and decisions at query time, on every query, forever. Extract at ingestion, dedupe against what is already there, and keep the metadata you will want to rank on later. That is the split between a harness that improves with use and one that just accumulates.

David Starmac Ai's profile picture
David Starmac Ai2 months ago

The harness part is underrated. In practice a good AGENTS.md changes results more than switching models. Question is how they kept guides from contradicting each other as they stack up.

palehonk's profile picture
palehonk2 months ago

harness turns an agent from a demo into infrastructure that's actually really cool

morph's profile picture
morph2 months ago

dang how somebody could let this be leaked? immediately saved this just in case xd

Kotte's profile picture
Kotte2 months ago

state lives in the harness not the model

zaiyanglab's profile picture
zaiyanglab2 months ago

Prompt tweaks help; evals on paying-user tasks decide who stays.

Andreas Edesberg's profile picture
Andreas Edesberg2 months ago

Where can I find the article?

Assaf Kipnis's profile picture
Assaf Kipnis2 months ago

I am loving that Google used Claude here and not Gemini

Jeremy RVN's profile picture
Jeremy RVN2 months ago

Matches what I see running agents in production. Model swaps stopped moving the needle a while ago. The wins come from constraints, verification steps and what the agent is allowed to touch. The harness is unglamorous work nobody benchmarks, which is exactly why it is the moat.

Yurii Filipchuk🇺🇦 — e/acc's profile picture
Yurii Filipchuk🇺🇦 — e/acc2 months ago

"the agent's ninety-five percent reliable." "how many steps?" "thirty." "so it's twenty-one percent reliable."

Aman yadav's profile picture
Aman yadav2 months ago

can you provide link of harness doc

Ivan Dorna's profile picture
Ivan Dorna2 months ago

Using AGENTS.md and SKILLS.md is THE problem.

Ryan Keary's profile picture
Ryan Keary2 months ago

how do you stop the guides from cancelling each other once the file gets long

stunspot | ⟨🤩⨯📍⟩ |'s profile picture
stunspot | ⟨🤩⨯📍⟩ |2 months ago

"We replaced prompt engineering! By writing prompts in _files_ not chat windows! You never need to worry about "prompting" again, except every time you write a tool description, SKILL.md, AGENTS.md, or want to understand context! Automate prompting without understanding it!"

techVector's profile picture
techVector2 months ago

Honestly that formula makes so much sense. We've been overthinking prompts when the real magic is in how you set up the whole system around the model. Same brain, better tools and rules. Feels obvious

Mildly Magical's profile picture
Mildly Magical2 months ago

@grok give me a brief bullet list of the key learnings for a solo Agentic dev he can put into practice.

Andry Dina's profile picture
Andry Dina2 months ago

Same Sonnet, better score from guides and external memory - that's the harness doing the work, not a new brain. When do we stop calling leaderboard jumps "model progress"?

Korens's profile picture
Korens2 months ago

saved

Chen's profile picture
Chen2 months ago

leaderboard drama never ends

拾光的 AI 应用观察's profile picture
拾光的 AI 应用观察2 months ago

Google这个思路我挺认同的,prompt engineering搞到后面确实边际效益递减,不如把约束外置到harness里。不过这种效率提升实际还得看具体任务,benchmark上好看不等于落地都能省token。AGENTS.md那套倒是跟我现在带团队做agent的流程有点像,markdown把规则写清楚比调prompt省心多了。

Blake Murphy's profile picture
Blake Murphy2 months ago

Most people judge an agent on its first session. The test that matters is the hundredth, when the model has forgotten everything and the harness has to carry the context, decisions, and guardrails across all of them. That is where an agent either becomes infrastructure or stays a demo.

Bally_AgenticAI's profile picture
Bally_AgenticAI2 months ago

Most builders stop at step 3 (the loop) - skip memory + observability and demos die in production. Chain harnesses together and cross-agent state consistency becomes the real bottleneck. #AgenticAI #ContextEngineering @LangChain

Related Videos

Harness vs. Graphs, clearly explained! a harness is great, and most people think it is the whole thing: retries, timeouts, a sandbox, a log, the context it assembles before every call. all of that is real work, and all of it wraps exactly one call. run it a hundred times and you have one call, made very safely, a hundred times. Graph engineering fixes this by moving the decision up a layer: not how safely one call is made, but which calls exist to be made at all. you need both, and here is the sentence that resolves the whole confusion: the harness is everything around one call. the graph is everything between them. ↳ around one call: retry, timeout, sandbox, log, assemble the context, hand back a result ↳ between calls: split, fan out, merge, gate, send back Prompts → Context → Harness → Loops → Graphs the harness does not go away when you build a graph. it moves under each node, and now there are five of them, each wrapping a call you would never have made by hand. the trick is knowing which layer a failure belongs to. turn a piece off and run it again. if the call still works, it was the harness. if the wrong step runs at all, it was the graph. people spend weeks hardening a harness around a node that should not have existed. one thing to know before you scale it. most of what people call their agent is a harness with a chat box on it. ↳ it retries, it times out, it logs, it assembles context, it holds one call up beautifully ↳ it has never once decided that a second call should exist, and that is the entire difference that last one catches careful people. a harness that never fails is not evidence the system is right. it is evidence one call went well, which is the smallest possible claim. and the one that eats whole nights: a harness cannot save you from the wrong step running. you can retry a bad decision three times with a clean log and perfect isolation, and all you bought was three copies of it. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓

Hanako

51,848 views • 2 months ago

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.

kocer

31,198 views • 2 months ago

Harness Engineering 2.0 is here! We benchmark models all the time but the harness running the model can dramatically improve the result, cost, and execution time too. And finally, devs just open-sourced Harness Arena, where you can test the same task across multiple harness and model combinations side by side. You provide one task and then run it across several model x harness configs that you want to test. Every config receives the same prompt, files, tools, skills, and evaluation checks. Harness Arena then returns: - total cost - execution time - whether the task passed - files and artifacts produced If you want to try it yourself, it is implemented within the open-source HarnessRouter project. Here's the repo: (don't forget to star it ⭐ ) HarnessRouter is a plug-and-play infrastructure layer that lets you run any harness under a single interface, like: - Codex - Hermes - Claude code - DeepSeek Harness - System One, powered by Jev - And 25 more agent harnesses In Harness Arena, the cheapest run does not automatically win. A config must complete the task correctly first. Cost and execution time are compared only among the configurations that pass. Here's the result from one Harness Arena test: - Hermes + GPT-5.2: 5/5 successful runs using 0.47 credits - Claude Code + Opus 4.8: 3/5 successful runs using 223 credits That is around 474x difference in credits, while the more expensive configuration also failed two runs. You can keep the model fixed to measure how much the harness changes the result. Or vary both to find the best harness and model pairing for that task. This helps answer practical questions such as: - Should this task run on Codex or Claude? - Can a cheaper config still pass every check? - Which setup is best to handle this workload? Once you find the best config, you can run future tasks through it using the same Unified Harness Protocol (UHP) interface for sessions, streaming, files, artifacts, cancellation, and failures. So you are no longer choosing an agent stack based only on a model leaderboard. Instead, you can benchmark the complete runtime on the task your product actually needs to solve. To dive deeper, my co-founder wrote an article explaining why model routing is different from harness routing, how UHP works, and what supporting multiple agent runtimes involves. It also covers the local setup, a working request, and how sessions and files move through the system. Read it below.

Avi Chawla

31,532 views • 7 days ago

Grok Bot + Kimi K3 can be turned into something bigger than an agent: an AI operating system the formula: AI OS = Router + Reasoning + Memory + Tools + Loops + Verification not one giant assistant. six layers that keep work moving without you step 1 -> Grok Bot becomes the operator. you give it the goal, it breaks the goal into jobs, assigns priorities and decides what part of the system should act next. step 2 -> Kimi K3 becomes the reasoning core. hard research, synthesis, long context and planning move here instead of forcing every task through the same model. step 3 -> externalize memory. store goals, decisions, failed attempts, artifacts and current state outside the chat. close the session, come back tomorrow, and the system still knows where it is. step 4 -> connect tools: search, code, files, APIs, docs and data. reasoning decides what should happen. tools actually make it happen. step 5 -> add the loop engine: plan -> execute -> inspect -> update memory -> retry. the loop can wait for new information, rerun a failed task, hand work to another agent or stop when the goal is complete. step 6 -> verify before output. tests, source checks, constraints and explicit completion rules decide whether the system ships the result or sends it back into the loop. that's the difference between an AI assistant and an AI operating system. an assistant waits for your next message. an operating system carries state, routes work and keeps moving. Grok Bot handles orchestration, Kimi K3 handles deeper reasoning, memory keeps the state alive, tools execute, the loop keeps the system running, verification decides when it is actually done. build one reliable loop and you have an agent. connect reasoning, memory, tools and multiple loops around it and you start building infrastructure. the full Grok Bot + Kimi K3 AI OS breakdown is below ↓

Alex

13,312 views • 2 months ago