Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Everyone's debating whether Opus or Codex or Gemini writes better code this month. Jyothi thinks that's the wrong question entirely. Models get replaced. Rankings flip every few months. Whatever wins the coding benchmark this quarter loses it next quarter, and the debate resets to zero. The harness doesn't reset.... show more

Aakash Gupta

295,027 subscribers

17,219 views • 3 months ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Jev founder, Diogo Almeida, just put out a PDF on building a Jev harness for coding agents the claim: 200x faster, 400x cheaper the model stopped being the bottleneck a while ago. the harness around it is where the cost and the speed actually live five decisions sit on every loop, and only one of them writes code hand this PDF and the article below to your Claude Code or Codex instance and let it rebuild its own setup your agent gets a better harness tonight, and monday looks different than it would have 👇

Jev founder, Diogo Almeida, just put out a PDF on building a Jev harness for coding agents the claim: 200x faster, 400x cheaper the model stopped being the bottleneck a while ago. the harness around it is where the cost and the speed actually live five decisions sit on every loop, and only one of them writes code hand this PDF and the article below to your Claude Code or Codex instance and let it rebuild its own setup your agent gets a better harness tonight, and monday looks different than it would have 👇

Hanako

37,887 views • 1 month ago

I'm sharing with you one of the best papers I've read on Harness Engineering it takes apart Claude Code (Opus 5.5), Codex, Gemini CLI and more, and boils all of them down to one line: agent = model + harness what you walk away with: > what a harness actually is > how today's coding agents are built under the hood > the patterns every one of them repeats > where the whole category is heading > their checklist for building your own the model gets the headlines. the harness decides whether the agent works loop and graph engineering on top of it - in my article below

I'm sharing with you one of the best papers I've read on Harness Engineering it takes apart Claude Code (Opus 5.5), Codex, Gemini CLI and more, and boils all of them down to one line: agent = model + harness what you walk away with: > what a harness actually is > how today's coding agents are built under the hood > the patterns every one of them repeats > where the whole category is heading > their checklist for building your own the model gets the headlines. the harness decides whether the agent works loop and graph engineering on top of it - in my article below

Mr. Buzzoni

84,277 views • 1 month ago

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness the promise: > 200x faster > 400x cheaper the model hasn't been the slow part for a while the speed and the cost sit in the harness wrapped around it • how to use it > drop this PDF and the article below into Claude Code or Codex > let it rebuild its own setup one evening of setup and your agent starts next week on a harness most teams still don't have

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness the promise: > 200x faster > 400x cheaper the model hasn't been the slow part for a while the speed and the cost sit in the harness wrapped around it • how to use it > drop this PDF and the article below into Claude Code or Codex > let it rebuild its own setup one evening of setup and your agent starts next week on a harness most teams still don't have

Mr. Buzzoni

80,191 views • 20 days ago

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness the promise: > 200x faster > 400x cheaper the model hasn't been the slow part for a while the speed and the cost sit in the harness wrapped around it • how to use it > drop this PDF and the article below into Claude Code or Codex > let it rebuild its own setup one evening of setup and your agent starts next week on a harness most teams still don't have 👇

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness the promise: > 200x faster > 400x cheaper the model hasn't been the slow part for a while the speed and the cost sit in the harness wrapped around it • how to use it > drop this PDF and the article below into Claude Code or Codex > let it rebuild its own setup one evening of setup and your agent starts next week on a harness most teams still don't have 👇

Mr. Buzzoni

128,732 views • 1 month ago

"The model is no longer the product. Codex, Perplexity Computer, or Claude Code - all are orchestration system. It takes a model and pairs it with an agent harness. What is an agent harness ? The rules for how the agent loops around" - Aravind Srinivas

"The model is no longer the product. Codex, Perplexity Computer, or Claude Code - all are orchestration system. It takes a model and pairs it with an agent harness. What is an agent harness ? The rules for how the agent loops around" - Aravind Srinivas

Rohan Paul

120,488 views • 4 months ago

The entire AI industry is racing to build the smartest model. Satya Nadella just admitted that is not where the money is. The model is not the product. The harness is. That is the exact line. And it changes what Microsoft is actually competing on. OpenAI, Anthropic, Google, xAI, Meta every frontier lab is pouring hundreds of billions into training compute, chasing the next capability jump. Each betting that raw model intelligence is the moat. Microsoft is doing the opposite. It is building the harness the orchestration layer that sits above the model, connecting it to tools, data, permissions, sub-agents, and enterprise workflows. And it is letting OpenAI, Anthropic, and MAI compete to plug into it. "You need the model. But the model is not the product. The harness is." So do the math on what a harness actually does. A raw model dropped into an enterprise answers questions. That is a chatbot. A harness turns that same model into an agent that reads the SharePoint, edits the ERP entry, pulls the GitHub PR, updates Salesforce, and files the Excel report with the right permissions, the right audit trail, and the right sub-agent for each sub-task. The model provides the intelligence. The harness converts intelligence into work. Now here's where it gets interesting. "Even the best model in the world will feel broken without a great harness. And an okay model with a great harness can feel like magic." If that is true, the enterprise buyer is not buying model quality. The enterprise buyer is buying the harness. Which means model quality becomes a commodity input over time, and harness quality becomes the sustainable moat. Compare that to the strategy the entire frontier lab industry is executing. Everyone else is chasing the numerator raw intelligence. Almost nobody at scale is racing to build the denominator the orchestration layer that determines whether that intelligence can actually be deployed profitably inside a real company. The frontier model race has a 10 to 20 percent chance of producing a single dominant winner. Nadella just told the industry he does not need to be that winner. If OpenAI wins, Microsoft wins. If Anthropic wins, Microsoft wins. If MAI wins, Microsoft wins. If someone Microsoft has never heard of trains a better model in 2027, Microsoft still wins. Because the compute they train on, the harness they get plugged into, the enterprise contracts they get delivered through, and the products they sit inside are all Microsoft. He is not building the best AI model. He is building the layer that the best AI model has to run on to make anyone money. I wonder which position looks more valuable in ten years.

The entire AI industry is racing to build the smartest model. Satya Nadella just admitted that is not where the money is. The model is not the product. The harness is. That is the exact line. And it changes what Microsoft is actually competing on. OpenAI, Anthropic, Google, xAI, Meta every frontier lab is pouring hundreds of billions into training compute, chasing the next capability jump. Each betting that raw model intelligence is the moat. Microsoft is doing the opposite. It is building the harness the orchestration layer that sits above the model, connecting it to tools, data, permissions, sub-agents, and enterprise workflows. And it is letting OpenAI, Anthropic, and MAI compete to plug into it. "You need the model. But the model is not the product. The harness is." So do the math on what a harness actually does. A raw model dropped into an enterprise answers questions. That is a chatbot. A harness turns that same model into an agent that reads the SharePoint, edits the ERP entry, pulls the GitHub PR, updates Salesforce, and files the Excel report with the right permissions, the right audit trail, and the right sub-agent for each sub-task. The model provides the intelligence. The harness converts intelligence into work. Now here's where it gets interesting. "Even the best model in the world will feel broken without a great harness. And an okay model with a great harness can feel like magic." If that is true, the enterprise buyer is not buying model quality. The enterprise buyer is buying the harness. Which means model quality becomes a commodity input over time, and harness quality becomes the sustainable moat. Compare that to the strategy the entire frontier lab industry is executing. Everyone else is chasing the numerator raw intelligence. Almost nobody at scale is racing to build the denominator the orchestration layer that determines whether that intelligence can actually be deployed profitably inside a real company. The frontier model race has a 10 to 20 percent chance of producing a single dominant winner. Nadella just told the industry he does not need to be that winner. If OpenAI wins, Microsoft wins. If Anthropic wins, Microsoft wins. If MAI wins, Microsoft wins. If someone Microsoft has never heard of trains a better model in 2027, Microsoft still wins. Because the compute they train on, the harness they get plugged into, the enterprise contracts they get delivered through, and the products they sit inside are all Microsoft. He is not building the best AI model. He is building the layer that the best AI model has to run on to make anyone money. I wonder which position looks more valuable in ten years.

Vikram M

21,463 views • 4 months ago

Harness Engineering 2.0 is here! We benchmark models all the time but the harness running the model can dramatically improve the result, cost, and execution time too. And finally, devs just open-sourced Harness Arena, where you can test the same task across multiple harness and model combinations side by side. You provide one task and then run it across several model x harness configs that you want to test. Every config receives the same prompt, files, tools, skills, and evaluation checks. Harness Arena then returns: - total cost - execution time - whether the task passed - files and artifacts produced If you want to try it yourself, it is implemented within the open-source HarnessRouter project. Here's the repo: (don't forget to star it ⭐ ) HarnessRouter is a plug-and-play infrastructure layer that lets you run any harness under a single interface, like: - Codex - Hermes - Claude code - DeepSeek Harness - System One, powered by Jev - And 25 more agent harnesses In Harness Arena, the cheapest run does not automatically win. A config must complete the task correctly first. Cost and execution time are compared only among the configurations that pass. Here's the result from one Harness Arena test: - Hermes + GPT-5.2: 5/5 successful runs using 0.47 credits - Claude Code + Opus 4.8: 3/5 successful runs using 223 credits That is around 474x difference in credits, while the more expensive configuration also failed two runs. You can keep the model fixed to measure how much the harness changes the result. Or vary both to find the best harness and model pairing for that task. This helps answer practical questions such as: - Should this task run on Codex or Claude? - Can a cheaper config still pass every check? - Which setup is best to handle this workload? Once you find the best config, you can run future tasks through it using the same Unified Harness Protocol (UHP) interface for sessions, streaming, files, artifacts, cancellation, and failures. So you are no longer choosing an agent stack based only on a model leaderboard. Instead, you can benchmark the complete runtime on the task your product actually needs to solve. To dive deeper, my co-founder wrote an article explaining why model routing is different from harness routing, how UHP works, and what supporting multiple agent runtimes involves. It also covers the local setup, a working request, and how sessions and files move through the system. Read it below.

Harness Engineering 2.0 is here! We benchmark models all the time but the harness running the model can dramatically improve the result, cost, and execution time too. And finally, devs just open-sourced Harness Arena, where you can test the same task across multiple harness and model combinations side by side. You provide one task and then run it across several model x harness configs that you want to test. Every config receives the same prompt, files, tools, skills, and evaluation checks. Harness Arena then returns: - total cost - execution time - whether the task passed - files and artifacts produced If you want to try it yourself, it is implemented within the open-source HarnessRouter project. Here's the repo: (don't forget to star it ⭐ ) HarnessRouter is a plug-and-play infrastructure layer that lets you run any harness under a single interface, like: - Codex - Hermes - Claude code - DeepSeek Harness - System One, powered by Jev - And 25 more agent harnesses In Harness Arena, the cheapest run does not automatically win. A config must complete the task correctly first. Cost and execution time are compared only among the configurations that pass. Here's the result from one Harness Arena test: - Hermes + GPT-5.2: 5/5 successful runs using 0.47 credits - Claude Code + Opus 4.8: 3/5 successful runs using 223 credits That is around 474x difference in credits, while the more expensive configuration also failed two runs. You can keep the model fixed to measure how much the harness changes the result. Or vary both to find the best harness and model pairing for that task. This helps answer practical questions such as: - Should this task run on Codex or Claude? - Can a cheaper config still pass every check? - Which setup is best to handle this workload? Once you find the best config, you can run future tasks through it using the same Unified Harness Protocol (UHP) interface for sessions, streaming, files, artifacts, cancellation, and failures. So you are no longer choosing an agent stack based only on a model leaderboard. Instead, you can benchmark the complete runtime on the task your product actually needs to solve. To dive deeper, my co-founder wrote an article explaining why model routing is different from harness routing, how UHP works, and what supporting multiple agent runtimes involves. It also covers the local setup, a working request, and how sessions and files move through the system. Read it below.

Avi Chawla

30,563 views • 3 days ago

The easiest way to run your agent harnesses using local models: Whether you're using Claude Code, Codex, OpenCode, or Pi, the local setup itself isn't really the hard part. The harder problem is figuring out which local model your machine can actually handle well. You have to think about RAM, model size, quantization, context length, KV cache, speed, accuracy, and a bunch of other trade-offs before you even start. Magnitude is an open-source solution that removes all of that guesswork for you. It profiles your machine, benchmarks what it can realistically run, recommends the best models for your hardware, and then lets you connect them to your preferred agent harness. The entire setup takes just two commands. In this video, I walk through the full process from hardware profiling to running a harness on a local model. Chapters: 00:00 Intro 00:17 What Magnitude is 00:45 Setup in two commands 01:29 Why picking a local model gets confusing fast 02:11 Hardware profiling + model recommendations 02:59 Choosing your harness 03:22 Connecting a harness that's already running 04:18 Outro Get started: (don't forget to star 🌟)

The easiest way to run your agent harnesses using local models: Whether you're using Claude Code, Codex, OpenCode, or Pi, the local setup itself isn't really the hard part. The harder problem is figuring out which local model your machine can actually handle well. You have to think about RAM, model size, quantization, context length, KV cache, speed, accuracy, and a bunch of other trade-offs before you even start. Magnitude is an open-source solution that removes all of that guesswork for you. It profiles your machine, benchmarks what it can realistically run, recommends the best models for your hardware, and then lets you connect them to your preferred agent harness. The entire setup takes just two commands. In this video, I walk through the full process from hardware profiling to running a harness on a local model. Chapters: 00:00 Intro 00:17 What Magnitude is 00:45 Setup in two commands 01:29 Why picking a local model gets confusing fast 02:11 Hardware profiling + model recommendations 02:59 Choosing your harness 03:22 Connecting a harness that's already running 04:18 Outro Get started: (don't forget to star 🌟)

Akshay 🚀

18,722 views • 1 month ago

this is pure f*cking gold for anyone running coding agents Jev founder, Diogo Almeida, wrote a guide on building a Jev harness the expensive model only writes the code. every fork around it goes to Jev the bill: > 6,000 forks a day on a frontier model: $180 > the same 6,000 forks on Jev: $0.25 > the whole harness, patches included: about $24 99 percent of what's left is the code itself, which is the only part you should be paying for what's inside: > the turn snapshot, five hooks, the cascade > thresholds read off your own receipts > the one rule a model never gets a vote on one evening of setup and your agent stops paying frontier prices to answer yes or no ↓

this is pure f*cking gold for anyone running coding agents Jev founder, Diogo Almeida, wrote a guide on building a Jev harness the expensive model only writes the code. every fork around it goes to Jev the bill: > 6,000 forks a day on a frontier model: $180 > the same 6,000 forks on Jev: $0.25 > the whole harness, patches included: about $24 99 percent of what's left is the code itself, which is the only part you should be paying for what's inside: > the turn snapshot, five hooks, the cascade > thresholds read off your own receipts > the one rule a model never gets a vote on one evening of setup and your agent stops paying frontier prices to answer yes or no ↓

Hanako

20,837 views • 1 month ago

One CLI. All models. Zero lock-in OpenClaude doesn't care if you use GPT, Kimi, MiniMax or Grok it just ships The model-agnostic, provider-free coding harness

One CLI. All models. Zero lock-in OpenClaude doesn't care if you use GPT, Kimi, MiniMax or Grok it just ships The model-agnostic, provider-free coding harness

GitLawb

13,063 views • 6 months ago

Google's team published a 9-page PDF on Harness Engineering. one formula replaced prompt engineering: Agent = Model + Harness the twist: same Claude Sonnet, same benchmark. change only the harness the playbook, six steps: step 1 -> add guides AGENTS.md, rule files, constraint docs. every line is a past agent failure turned into a permanent fix step 2 -> add sensors linters, tests, validation scripts the agent runs on its own output before a human sees it step 3 -> build the agentic loop plan, execute, verify, fix. bounded retries, budget caps, escalation when stuck step 4 -> externalize memory the model forgets every session. the harness holds state, decisions and artifacts across all of them step 5 -> enforce permissions which tools, how many writes, what needs approval. safety lives in the harness, never in the model step 6 -> wire observability track every tool call, cost and retry. trip wires fire when behavior drifts the result: your agent turns from a demo into infrastructure every failure makes the system permanently better, not just the next conversation that's also the line between an agent you show people and one a client pays you to leave running one harness gets you a reliable loop. connect several and it becomes graph engineering, where one agent's failure turns into another agent's sensor the PDF stops at the harness. the graph is the next layer that breakdown, running on Kimi, is below ↓

Google's team published a 9-page PDF on Harness Engineering. one formula replaced prompt engineering: Agent = Model + Harness the twist: same Claude Sonnet, same benchmark. change only the harness the playbook, six steps: step 1 -> add guides AGENTS.md, rule files, constraint docs. every line is a past agent failure turned into a permanent fix step 2 -> add sensors linters, tests, validation scripts the agent runs on its own output before a human sees it step 3 -> build the agentic loop plan, execute, verify, fix. bounded retries, budget caps, escalation when stuck step 4 -> externalize memory the model forgets every session. the harness holds state, decisions and artifacts across all of them step 5 -> enforce permissions which tools, how many writes, what needs approval. safety lives in the harness, never in the model step 6 -> wire observability track every tool call, cost and retry. trip wires fire when behavior drifts the result: your agent turns from a demo into infrastructure every failure makes the system permanently better, not just the next conversation that's also the line between an agent you show people and one a client pays you to leave running one harness gets you a reliable loop. connect several and it becomes graph engineering, where one agent's failure turns into another agent's sensor the PDF stops at the harness. the graph is the next layer that breakdown, running on Kimi, is below ↓

Annatar.md

79,906 views • 2 months ago

Most agent comparisons end up comparing models. We wanted to test the other half: the harness. That's the environment that decides whether a search actually runs, a source gets checked, and a file gets delivered. So we gave Minara, OpenClaw, Hermes and Claude Code the same tasks on the same base model: charts, research, code, and a prompt-injection test. Same model. Different outcomes. In the video, one AppLovin research note, GPT-5.6 Luna on both sides. Minara Harness delivered it in 1:51. Codex was still working when the recording ended. (5x speed) Full case studies 👇

Most agent comparisons end up comparing models. We wanted to test the other half: the harness. That's the environment that decides whether a search actually runs, a source gets checked, and a file gets delivered. So we gave Minara, OpenClaw, Hermes and Claude Code the same tasks on the same base model: charts, research, code, and a prompt-injection test. Same model. Different outcomes. In the video, one AppLovin research note, GPT-5.6 Luna on both sides. Minara Harness delivered it in 1:51. Codex was still working when the recording ended. (5x speed) Full case studies 👇

Minara AI

27,006 views • 1 month ago

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness what their tests show: > up to 200x faster > up to 400x cheaper the model hasn't been the bottleneck for a while speed and cost both come from the harness you wrap around it • how to use it > give this PDF and the article below to Claude Code or Codex > ask it to rebuild its own setup around Jev • how it turns into a promotion > pick the agent workflow your team pays the most for > rebuild it in one evening > bring the before and after bill to your manager one evening of Jev engineering, and next week you're the one running a harness most teams haven't built yet

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness what their tests show: > up to 200x faster > up to 400x cheaper the model hasn't been the bottleneck for a while speed and cost both come from the harness you wrap around it • how to use it > give this PDF and the article below to Claude Code or Codex > ask it to rebuild its own setup around Jev • how it turns into a promotion > pick the agent workflow your team pays the most for > rebuild it in one evening > bring the before and after bill to your manager one evening of Jev engineering, and next week you're the one running a harness most teams haven't built yet

Annatar.md

73,481 views • 28 days ago

Why harnesses need to be neutral “You want to be able to switch models. The harness can be very tied to the model, or it can be neutral. The harness’s job is to take the model and interact with everything else, including data, APIs and memory. If you build memory in a way that is linked to the harness, and the harness is linked to the model, then you are kind of stuck.” Harrison Chase Love to hear your thoughts Charles Packer Hamel Husain swyx

Why harnesses need to be neutral “You want to be able to switch models. The harness can be very tied to the model, or it can be neutral. The harness’s job is to take the model and interact with everything else, including data, APIs and memory. If you build memory in a way that is linked to the harness, and the harness is linked to the model, then you are kind of stuck.” Harrison Chase Love to hear your thoughts Charles Packer Hamel Husain swyx

Harry Stebbings

23,760 views • 20 days ago

Guys, stop underestimating the harness. GPT-5.6 Sol through Claude Code vs Codex produced a GIGANTIC difference in my benchmark. But Opus 5 is the real innovation: High, xHigh, and Max don’t just feel like different thinking levels—they feel like COMPLETELY DIFFERENT MODELS. It’s basically like getting three or four Opus models in one.

Guys, stop underestimating the harness. GPT-5.6 Sol through Claude Code vs Codex produced a GIGANTIC difference in my benchmark. But Opus 5 is the real innovation: High, xHigh, and Max don’t just feel like different thinking levels—they feel like COMPLETELY DIFFERENT MODELS. It’s basically like getting three or four Opus models in one.

OmedTheVibeCoder

36,074 views • 3 months ago

holy sh*t, Google Research just open-sourced an agent that can improve its own harness steal this blueprint for a self-improving agent the prompts, memory, skills and tool guidance around your model can evolve as it works it’s called Harness² the loop: → propose changes to the agent’s setup → run candidate setups on the same task → compare what actually happened → compose a refined harness from that evidence → carry useful lessons into future tasks the clever part: it refines the harness on unfamiliar tasks without a ground-truth answer to guide those edits the architecture: propose → probe → compose → retain lessons → repeat the part builders should steal: treat your agent’s setup as something you can experiment on pick a recurring failure propose a specific harness change compare execution traces on the same task evaluate the result keep the evidence with the change the repo includes OpenCode and Codex executors, parallel and sequential refinement, and a local demo you can run without model credentials save this, then read the full guide for Opus 5.5 Harness Engineering ↓

holy sh*t, Google Research just open-sourced an agent that can improve its own harness steal this blueprint for a self-improving agent the prompts, memory, skills and tool guidance around your model can evolve as it works it’s called Harness² the loop: → propose changes to the agent’s setup → run candidate setups on the same task → compare what actually happened → compose a refined harness from that evidence → carry useful lessons into future tasks the clever part: it refines the harness on unfamiliar tasks without a ground-truth answer to guide those edits the architecture: propose → probe → compose → retain lessons → repeat the part builders should steal: treat your agent’s setup as something you can experiment on pick a recurring failure propose a specific harness change compare execution traces on the same task evaluate the result keep the evidence with the change the repo includes OpenCode and Codex executors, parallel and sequential refinement, and a local demo you can run without model credentials save this, then read the full guide for Opus 5.5 Harness Engineering ↓

beamnxw ./

12,039 views • 17 days ago

$HARNESS vs LOOP vs GRAPH ➜ STOP MIXING THEM UP Most people treat these three as the same thing. They’re not 1\ Harness = the machinery around the model (tools, state, permissions, memory, sandboxes, observability) 2\ Loop = the repeated work + evidence + feedback cycle with clear stop rules 3\ Graph = the explicit topology (nodes, branches, joins, controlled cycles) Clean mental model environment → feedback → flow > A raw model can’t maintain state, run tests, or restart failed jobs. Those capabilities come from the harness > Loops turn one-shot calls into managed processes that only stop when evidence proves success > Graphs decide which component is allowed to run next Most production failures that get blamed on the model are actually failures of harness, loop, or graph design Design the three layers together. Diagnose by layer when something breaks Guide in the article below$

HARNESS vs LOOP vs GRAPH ➜ STOP MIXING THEM UP Most people treat these three as the same thing. They’re not 1\ Harness = the machinery around the model (tools, state, permissions, memory, sandboxes, observability) 2\ Loop = the repeated work + evidence + feedback cycle with clear stop rules 3\ Graph = the explicit topology (nodes, branches, joins, controlled cycles) Clean mental model environment → feedback → flow > A raw model can’t maintain state, run tests, or restart failed jobs. Those capabilities come from the harness > Loops turn one-shot calls into managed processes that only stop when evidence proves success > Graphs decide which component is allowed to run next Most production failures that get blamed on the model are actually failures of harness, loop, or graph design Design the three layers together. Diagnose by layer when something breaks Guide in the article below

beamnxw ./

19,595 views • 3 months ago

Tommy discusses the push for companies to own the full stack around their AI models. “Everything around the model should be yours: the orchestration, the memory, the model weights, the harness, the company data, the permissions, the approvals, the users.”

Tommy discusses the push for companies to own the full stack around their AI models. “Everything around the model should be yours: the orchestration, the memory, the model weights, the harness, the company data, the permissions, the approvals, the users.”

Delphi Digital

38,519 views • 3 months ago

I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard. Which harness should you run your open model in? I ran 28,849 trials to find out. The leaderboard compares nine coding-agent harnesses on open models. In each comparison only the harness changes: the model, provider and reasoning effort stay fixed, and every task gets three attempts. The trials ran GLM-5.3-Flash, DeepSeek-V4.1-Flash and Flint on Terminal-Bench 2.1 and DeepSWE, for $4,746 of inference. Averaged over the three models, mini-swe-agent (Kilian Lieret), a bash-only agent, solves 51.4% of tasks and OpenClaw🦞 27.3%. Four others are statistically tied with mini-swe-agent. One of them is Pi (Pi), at 49.4% for $0.29 per solved task, about 45% less than mini-swe-agent's $0.53. The best harness depends on the model: mini-swe-agent on GLM-5.3-Flash, Hermes Agent (Nous Research) on DeepSeek-V4.1-Flash and Kimi Code (Kimi.ai) on Flint. OpenCode costs $3.35 per solved task, 11.7 times as much as Pi. See Harness Hub's Harness Lens for what the nine harnesses do differently. It documents the prompts, tools and compaction of 34 coding-agent harnesses and links every claim to its source. Leaderboard: Harness Lens:

I burned a shitload of compute so you don't have to. Releasing the Harness Leaderboard. Which harness should you run your open model in? I ran 28,849 trials to find out. The leaderboard compares nine coding-agent harnesses on open models. In each comparison only the harness changes: the model, provider and reasoning effort stay fixed, and every task gets three attempts. The trials ran GLM-5.3-Flash, DeepSeek-V4.1-Flash and Flint on Terminal-Bench 2.1 and DeepSWE, for $4,746 of inference. Averaged over the three models, mini-swe-agent (Kilian Lieret), a bash-only agent, solves 51.4% of tasks and OpenClaw🦞 27.3%. Four others are statistically tied with mini-swe-agent. One of them is Pi (Pi), at 49.4% for $0.29 per solved task, about 45% less than mini-swe-agent's $0.53. The best harness depends on the model: mini-swe-agent on GLM-5.3-Flash, Hermes Agent (Nous Research) on DeepSeek-V4.1-Flash and Kimi Code (Kimi.ai) on Flint. OpenCode costs $3.35 per solved task, 11.7 times as much as Pi. See Harness Hub's Harness Lens for what the nine harnesses do differently. It documents the prompts, tools and compaction of 34 coding-agent harnesses and links every claim to its source. Leaderboard: Harness Lens:

Joël Niklaus

218,063 views • 1 day ago

This is pure f*cking treasure Google researchers just open-sourced an agent that rebuilds its own harness on tasks it has never seen The model stays frozen, not a single weight changes. Everything around it gets better instead Here's how one loop works: > The agent runs the task with its current harness > A Harness Editor proposes edits: general ones and domain-specific ones > It can touch 8 parts: system prompt, guardrails, memory, skills, plugins, tool guidance, scripts and subagents > Every edit gets probed on the same task > A Harness Composer compares how each version behaved and merges what worked into the final harness > An Experience Reflector writes down what it learned > Those lessons go into a persistent Improvement Harness that guides the next task No ground-truth labels, nobody tells it the right answer It just keeps getting better, task after task

This is pure f*cking treasure Google researchers just open-sourced an agent that rebuilds its own harness on tasks it has never seen The model stays frozen, not a single weight changes. Everything around it gets better instead Here's how one loop works: > The agent runs the task with its current harness > A Harness Editor proposes edits: general ones and domain-specific ones > It can touch 8 parts: system prompt, guardrails, memory, skills, plugins, tool guidance, scripts and subagents > Every edit gets probed on the same task > A Harness Composer compares how each version behaved and merges what worked into the final harness > An Experience Reflector writes down what it learned > Those lessons go into a persistent Improvement Harness that guides the next task No ground-truth labels, nobody tells it the right answer It just keeps getting better, task after task

NO1ennn

19,359 views • 7 days ago