this is f*cking gold Google engineers explained how to... make AI rewrite an agent’s system prompt against failing tests. a failed check becomes the next repair job. the regression suite watches for whatever that repair breaks. even the instructions become something you can test and improve. Agent = Model + Harness. the independently compiled page here maps the broader system into six parts: guides carry project rules, constraints and lessons from past failures > sensors check the work through tests, linters and validators > the loop runs the task, checks the result, retries within limits and escalates > memory preserves state, artifacts and decisions across runs > permissions control tools, writes and actions requiring approval > observability records what happened, what it cost and where it failed an agent changes a build file and announces “done.” the check looks for evidence that it actually ran the validator. if that behavior is missing, you have a specific failure to target. adjust the instructions. rerun the evaluations. check whether the improvement holds without breaking existing behavior. now a prompt change has a test history. a recurring mistake has a regression check. the next model upgrade has something concrete to pass. bookmark the diagram. give “done” a test it has to pass.show more

If you are trying to understand where AI agents... are going, learn harness engineering. A capable model is only one part of an agent system. Once the model begins reading files, calling tools, modifying state and working across many steps, the quality of the system depends increasingly on the software around it. Consider a coding agent working through a large repository. The model can decide that it needs to inspect a file, search for a symbol, make an edit or run a test, but those decisions do not execute themselves. The surrounding runtime has to decide which resources are available, whether the requested action is permitted, how the operation should be performed, what result should be retained, and what information should be presented to the model on the next step. This becomes harder as the run gets longer. As history accumulates, replaying everything can become costly and less effective. The harness has to decide what should remain in context, what should be summarized or retrieved later, and what belongs in persistent state outside the context window. Execution has similar problems. A long-running agent may need to survive an interruption, avoid repeating completed work, enforce permissions around consequential actions, and preserve enough history to reconstruct what happened when the final result is wrong. These are harness problems. The harness is the layer that manages context, tools, execution, state, checkpoints, limits and traces around the model. Harness engineering is the work of designing and improving that layer. Engineers inspect execution traces, evaluate agents on representative tasks, look for recurring failure modes, and then change things such as context selection, tool interfaces, state handling or execution controls. That last part matters because agent failures are often not fixed by changing the model. Sometimes the useful change is in what the model sees, how a tool is exposed, what state is preserved, or what the runtime does after a failed step. As agents take on longer tasks, the demands on this surrounding software grow. Model capability remains essential, but harness engineering is what turns that capability into an execution process that can be controlled, inspected, tested and improved.show more

this AI engineering breakdown from IBM Technology is pure... f*cking treasure it explains the role as a connected system 01 Python and APIs give you the surface to build on: services, data, integrations, and the software around the model 02 RAG and embeddings solve the context layer: finding the right information before the model has to reason about it 03 agents turn that context into work: they choose tools, run steps, and move toward an outcome 04 observability closes the feedback loop: traces, metrics, failures, latency, and the evidence needed to improve the system 05 deployment is where the system becomes real: reliability, constraints, and the conditions that let people actually depend on it the model sits inside this architecture the AI engineer connects every layer into one operating loop: build → retrieve → act → observe → ship save this, then build your solo business with an AI employee ⭣show more

i don't f*cking understand why this isn't popular yet... someone has created a memory system that uses 90% fewer tokens while still finding all the expected symbols it builds a local graph of source symbols and their relationships. architecture, decisions and handoffs live as Git-tracked Markdown. a note can point to the code behind it, so a code change can flag that knowledge for review the loop: scan the repo → map the code → write the why → anchor notes to symbols → retrieve a task-sized slice → review drift → commit to Git → continue in the next session in the author's small, one-repo test, graph retrieval returned 10.74× less context than grep top-3 while finding every expected symbol across six tasks the useful part is the connection between memory and evidence. the repo carries what the agent learned, and the code gives you a way to check whether that knowledge still holds save this, then build a second brain for your company⭣show more

Alright, now that we know *what* an agent is,... how does it actually work? When you ask for help on a task, the agent plans a series of steps and executes them directly in the application on your behalf, using the tools it has access to. Say you are booking a local service or trying to organize your inbox (which typically takes multiple steps): the AI model first plans how to achieve the task using its existing knowledge and then interacts with your inbox to execute the task. The agent will continue until it is confident the task has been successfully completed.show more

Kimi K3 + GPT-6 Astra can become something bigger... than two agents: a Two-Brain AI Operating System the formula: Two-Brain OS = Research Brain + Execution Brain + Shared State + Router + Tools + Verification not two models doing the same job. two specialized brains connected through one persistent system step 1 -> Kimi K3 becomes the research brain. complex research, source comparison, long-context synthesis and strategic planning happen here. its job is to explore the problem and compress the findings into a clear plan. step 2 -> GPT-6 Astra becomes the execution brain. it receives the plan, writes code, operates tools, creates artifacts and turns decisions into finished work. step 3 -> build a shared state layer. store the objective, evidence, decisions, constraints, failed attempts and next action outside the chat. both brains always know what happened and where the system should continue. step 4 -> add the router. uncertainty and open-ended questions move to Kimi K3. execution, tool use and deterministic tasks move to GPT-6 Astra. every job reaches the brain designed to handle it. step 5 -> create the handoff loop: research -> plan -> execute -> inspect -> update state -> continue. if execution reveals missing information, the task returns to Kimi. if the plan is ready, Astra takes control again. step 6 -> verify before completion. tests, source checks, constraints and explicit success criteria decide whether the result ships or returns to the correct brain with a clear failure signal. that is the difference between using two AI models and building a Two-Brain AI Operating System. Kimi K3 expands the search space. GPT-6 Astra converts it into action. shared state preserves progress, the router controls every handoff, tools execute the work and verification decides when the system is actually finished. one model can generate an answer. two specialized brains connected through memory, routing and verification can run an entire workflow. the full Kimi K3 + GPT-6 Astra Two-Brain OS breakdown is below ↓show more

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE... ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.show more

Don't train the model, evolve the harness. I read... a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.show more

this is f*cking beyond comprehension. Google engineers just shipped... the entire agent lifecycle in one release: build, scale, govern. and every piece answers a specific way agents die in production > context layers (Static, Turn, User, Cache): you decide what the model carries between turns, so token spend stops being a mystery > a self heal plugin: the agent notices a tool call failed and retries it a different way instead of dying mid run > adk deploy: one command from your laptop to the managed runtime, no packaging, no infra ticket > Go joins Python and Java, with its own A2A SDK then the part nobody builds for themselves: > a dashboard on token consumption, latency, error rates and tool calls: the four things that actually kill an agent > a traces tab that opens the real sequence of actions the agent took, step by step > a playground wired to the deployed agent, past sessions included, so debugging is not a redeploy loop > an Evaluation Layer with a User Simulator, because you cannot unit test a non deterministic system and the part that decides whether it ever ships: > agents get native identities as first class IAM principals: least privilege applies to them like it does to people > Model Armor screens prompt injection, tool calls and responses, inline for Gemini or over REST > Security Command Center inventories every agentic asset and flags data exfiltration by an agent ADK is already at 7 million downloads. the runtime has a free tier, and express mode runs off a Gmail address. the prototype was never the hard part.show more

Stanford researchers did it again. They just built the... agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. Akshay wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. Read it below.show more

Let me explain the agent loop, simple It's the... core of every agentic system, and the part most people overcomplicate It's just this: 1. Send messages to the model 2. Model responds, maybe calls a tool 3. You run the tool 4. Append the result back to messages 5. Repeat until stop_reason is end_turn Step 4 is the whole thing, the write-back is what makes it an agent The model has to see what actually happened before it decides the next move That's the entire loop... understand this cold before you reach for a frameworkshow more

What happens when an AI stops telling you how... to build something and actually builds it? Nex-N2.5 Pro took an idea from concept to playable game: → Rough concept → 2D design → Blender 3D environment → Working game The interesting part wasn’t the final result. It was the process. The model interacted directly with the tools, observed the changes on screen, made adjustments, and continued working instead of simply generating instructions. That’s the real computer-use test. Can an AI: • Open Blender and build a scene? • Inspect the output and iterate? • Write a script, tweak parameters, and troubleshoot? • Hit a limitation, switch to the GUI, and finish the task? Nex-N2.5 Pro is starting to make that workflow feel surprisingly real. You can try it on OpenRouter for a limited time: Try Nex-N2.5 Pro on OpenRouter #NexN2 #Blender #ComputerUse #OpenRoutershow more

I went a little overboard with Codex last week... and burned through my entire weekly allowance in two days. Luckily, my quota reset today. Otherwise, I’m not sure what I would’ve done. It got me thinking: instead of asking one large model to handle everything from start to finish, why not let a stronger model plan the project and review the work, while a model built for execution handles the day-to-day implementation? So I tried it. The result was better than I expected. I used GPT-5.6 Sol in Codex as the decision-maker, then ran Ling-3.0-flash from Ant Ling inside OpenCode as the execution engine. Together, they built a small 3D farming game. Before writing any code, I had Codex create four documents: SPEC.md defined the product scope and the lines we couldn’t cross. ARCHITECTURE.md laid out the isometric coordinate system, state machine, and module boundaries. TASKS.md broke the project into small jobs Ling could tackle one at a time. ACCEPTANCE.md explained how each step would be tested and what “done” actually meant. Then I gave Ling a very straightforward role: You are the execution model for this project. Read all four documents before you begin. Work only on the task assigned for this round. When you’re done, run typecheck, test, and build. If anything fails, read the error, fix it, and run the checks again. Do not move on to the next task early. Ling handled dependency installation, project structure, strict TypeScript configuration, test setup, and a production build in 6 minutes and 3 seconds. It ran into issues with the Vite test config, a TS6310 error, and a missing jsdom dependency along the way. Instead of stopping at the first error, it kept reading the logs and fixing the problems until all three checks passed. The speed was honestly hard to believe. If you exclude the time spent waiting on tools, it was producing more than 100 tokens per second. That made the whole development loop feel noticeably faster. After this experiment, I’m planning to keep using the same workflow. If the task is small, there’s no reason to call an expensive planning model for every single step. If the task is large, handing the entire project to a Flash model in one prompt isn’t a great idea either. The setup that makes more sense to me is: Use a more capable model such as Codex to explore the project, make architectural decisions, and break the work down. Put the constraints into specs, schemas, types, and tests instead of leaving them buried in chat history. Give Ling-3.0-flash a steady stream of clear, verifiable implementation tasks. Report bugs with structured context and actual error logs, rather than saying, “It still doesn’t work.” Bring Codex back in for architecture reviews, visual checks, and changes that affect multiple parts of the project. The point of this setup isn’t to give AI a big “build the whole project” button. It’s to turn software development into a pipeline with a much more sensible cost structure: Codex figures out the plan, sets the boundaries, and catches problems. Ling-3.0-flash moves quickly, calls tools reliably, and works through well-defined tasks at scale. For agent workflows that involve lots of repetitive edits, production tasks, and tool calls, this may be a more practical answer than simply using the biggest model for everything.show more

𝗧𝗵𝗲 𝟰 𝗟𝗮𝘆𝗲𝗿𝘀 𝗼𝗳 𝗮𝗻 𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺 An agent... can burn tokens, declare success, and still fail the tests. That’s often an architecture problem, not a prompting problem. 𝟭. 𝗟𝗼𝗼𝗽 → Decides whether to continue Acts → verifies → retries until there’s evidence of success. 𝟮. 𝗚𝗿𝗮𝗽𝗵 → Decides what runs next Handles branches, retries, handoffs, fallbacks, and shared state. 𝟯. 𝗛𝗮𝗿𝗻𝗲𝘀𝘀 → Gives the model its capabilities Tools, APIs, files, memory, permissions, context, and execution environment. 𝟰. 𝗠𝗲𝘁𝗮-𝗛𝗮𝗿𝗻𝗲𝘀𝘀 → Governs multiple agent environments Coordinates different agents, tools, policies, permissions, and workflows. 𝗧𝗵𝗲 𝗺𝗲𝗻𝘁𝗮𝗹 𝗺𝗼𝗱𝗲𝗹: Loop = verification Graph = workflow Harness = capability Meta-harness = governance A better model can improve reasoning. But reliable agents depend on the architecture built around the model.show more

Stop manually making flowcharts. Try this GitHub skill with... Claude Code. I found an open-source tool that turns a plain description of a software system into a clean, interactive diagram. It's called Archify, and it has more than 66,000 stars on GitHub. You add it as a skill to an AI coding agent like Claude Code, Codex, Cursor, or OpenCode. Then you describe the system in the chat in one line. Archify turns that line into a single HTML file you can open, click through, and send to anyone. If you point your agent at an existing codebase, it reads the code and Archify maps how the pieces connect. I liked five things about it. 1. It makes five kinds of diagrams, including system maps and step-by-step workflows. 2. You can click any piece and trace what connects to it. 3. It checks the layout before it hands the diagram over, so arrows don't pile up and labels don't overlap. 4. It has dark and light themes and exports to PNG, SVG, or a short video. 5. You keep editing in plain chat, with requests like "add Redis" or "highlight the rollback path." You don't need a codebase to use it. A description in the chat is enough.show more

AGENT ARCHITECTURE ROUTES WORK. IT DOES NOT REMEMBER WORK.... THAT GAP IS WHY YOUR LOOP KEEPS FIXING THE SAME BUG TWICE. these are two different engineering problems. every agent that silently drifts is missing one of them. architecture answers what runs. harness → loop → graph. it defines the tools, the retries, the branching routes, the approval gates. context ops answer what the run knows. write → read → compress → isolate. it defines what gets saved between attempts, pulled in on read, summarized on overflow, and split across sub-agents. for two months i believed a solid harness plus a verifier loop was enough. my coding agent kept re-discovering the same test failure across retries. the loop was working. it just had nowhere to write what it had already learned. here is the decision rule: if your agent forgets across restarts, add write and read. if it stalls on long tasks, add compress. if two sub-agents step on each other, add isolate. architecture without context ops is a well-routed system with amnesia.show more

sorry, they just did WHAT someone gave a machine... one disease name, the leading cause of blindness in the developed world with 1.5 million americans already in its path, and it came back pointing at a drug that has sat in pharmacies for years under a different label: 551 papers read in 30 minutes against the 294 hours a human would have needed, and the loop that did it is public on GitHub most agent setups answer one question at a time, so the ceiling on the work is the quality of the question you happened to think of this one was handed a single question and wrote the second one itself. turns out that follow-up is where the real find was: a target called ABCA1, upregulated threefold, in an experiment no human ordered i read the whole paper looking for the trick, and the trick is structural. that is the second question, and it is the gap between an assistant and a factory: - hand the loop a field rather than a task: it was given a disease, and choosing the mechanism was part of its job - make it rank before it spends: 151 papers in, ten candidate mechanisms out, scored against each other before anything touched a bench - split reading from judging, so the agent that forms the theory is a different agent from the one grading it - close every cycle on physical reality: the verdict was an experiment, and another model's opinion was never allowed to stand in for one - feed each result back as the next question rather than a log line, which is the step almost nobody builds - search what already passed inspection first: the winner was an approved compound with a safety file already on record - write down what the round learned before opening the next one, so round two starts where round one stopped my read, and i think it is the uncomfortable one: reading was the entire bottleneck in that field, and everybody spent the decade optimising the writing. people ran every physical experiment here, the analysis agent needs a domain expert writing its prompts, and the authors decline to call this the leap it resembles. the thinking got replaced, and the hands did not so the question i cannot answer for my own setup: which step of your loop still stops dead until you sit down and type something bookmark this one. the four parts that turn one model into a line that runs like this, the queue, the rooms, the write permissions and the gate, are built file by file in the piece below ↓show more

Google has just launched an AI feature capable of... planning, reasoning and searching the web. All this is done in a single prompt using Deep Research. This is literally the first reasoning model that has access to the internet. Game-changer. How to use it below 🧵show more

Anthropic shouldn't have revealed this plugin. Its name speaks... for itself "You should know" Its capability: Opus 5.5 writes, the second agent reads, garbage aside: you get only important and reliable information Opus 5.5 can run continuously for 60 minutes, generating long summaries that nobody checks And this is what many miss: > a test it skipped "for now" > a file it modified even though you didn't ask for it > a warning "this might break the system" hidden somewhere at the bottom All of this is written there. You just never saw it Now another AI checks the text for you This is an automatic feature Enabled with a single line: ⁠/plugin enable cc-plugin-you-should-know@builtin⁠ Do this when you start working: > Open Claude Code > Paste the line above > Give Opus a long task and see how it analyzes the text and eliminates obvious errors Now the agent won't waste your money: you will get the expected result Bookmark thisshow more

You've probably scrolled past a dozen posts about Jev... this week without anyone telling you what it actually is. It's the first model from TypeSafe, a lab started by one of the researchers behind ChatGPT. It's a decision engine: you give it options, it picks one and tells you how sure it is. It cannot write a single word, and that is the interesting part. Every other AI you use writes. That is the whole interface. So when software needs a plain yes or no, we make a model write a paragraph and then dig the answer back out of it. Fine in a chat window where a human reads it. Bad inside software, where code has to act on it. The bet is that the valuable half was never the writing. It was the deciding. That problem showed up in WordPress years ago, and it is the reason WPVibe works the way it does. The AI does the work. Anything permanent stops and waits, because a delete that skips the trash is not something software should decide on its own.show more

this is f**king dangerous someone figured out how to... make Opus 4.8 run on Fable 5's brain with one prompt access to the best model is never guaranteed. It disappeared once already this year. but you can use it forever. here's how: 1. ask Fable 5: "write the operating manual your replacement will run on" (procedures, failure modes, a 5-question self-test) 2. save the output as one .md file and drop it into a new Claude Project as the project instructions 3. switch to Opus 4.8 and now your everyday model runs off the smart one's method, no top-tier price save and bookmark this no matter what full extraction prompt is in the article below: ↓show more