Загрузка видео...

Не удалось загрузить видео

На главную

At Uber, our testing agent hit an outage in Australia that I still think about. We had the recording. That's about it. No network trace for the run. No logs mapped to each step. No perf data. Just a video of an agent pressing a button and a team...

42,149 просмотров • 2 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. Akshay wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. Read it below.

Avi Chawla

441,974 просмотров • 3 месяцев назад

I just built a Meta Ads diagnostic in Claude Code that tells you WHY your account broke, not just what changed 🤯 It spins up a team of agents that each investigate a different reason performance dropped, then argue against each other to kill the wrong answer before it ever reaches you. All inside Claude Code. Perfect for DTC brands and agencies who panic-kill creative the second CPA spikes. If you've watched ROAS fall off a cliff and opened Ads Manager with ten tabs going, you already know what happens next. Your gut says "creative fatigue." You kill your best-performing ad. A week later performance is still broken, because that was never the problem. Guessing wrong is the most expensive move in paid social. This workflow ends the guessing: → One agent investigates each competing theory — creative fatigue, budget and delivery changes, traffic quality, offer and seasonality → Each one is blind to the others, reasoning only from its own slice of the data so they can't bias each other → A refuter agent then attacks every surviving theory and tries to kill it → A theory only stands if the data can't disprove it → You get a ranked diagnosis: the real cause, the evidence for and against it, and the one move to make this week No anchoring on the first obvious answer. No killing winning creative on a hunch. No "here's what happened" reports that never tell you why. What you get: → Every theory tested in parallel instead of one biased guess → An adversarial pass that kills the wrong answer before you act on it → A ranked diagnosis with confidence levels and evidence both ways → A reusable workflow you drop next month's export into and re-run Built 100% in Claude Code with the new dynamic workflows. The first account I ran it on looked like textbook creative fatigue. The workflow disagreed, and traced the real cause to a budget change that had doubled spend and flooded delivery with junk traffic. I put together a full playbook with the exact workflow, the prompt, and how to run it on your own account. Want it for free? > Like this post > Comment "META" And I'll send it over (must be following so I can DM)

Mike Futia

12,884 просмотров • 4 месяцев назад

this is f*cking gold Google engineers built an agent that tests thousands of search strategies for the cost of running one, by letting it dream inside its own memory. it is called Dream-RSI. Google, Google DeepMind, Maryland, Virginia. 17 authors. the whole thing rests on one observation: a finished run is not a log. it is a map of a world that already exists. > every discovery run records a tree. each node is one attempt with its full result stored > that tree becomes a replay simulator. the paper calls them worlds > the agent dreams inside it: thousands of alternative strategies, testing different branches, different orders, different concurrency, different stopping rules > every one of those evaluations costs zero execution. nothing is ever run twice > the winner goes back online, produces a new tree, and the pool of worlds grows the coding agent underneath is never touched. this is an orchestration layer sitting on top of whatever you already run. what it bought them, against the same agent with a frozen strategy: > Lasso: 3587ms down to 2931ms, on 317 agent calls instead of 550 > on Flash: 2517ms to 2351ms, 1879 calls instead of 3200 > both solvers beat sklearn and glmnet on all six held-out datasets > GPU kernels: the same result on 2.43x and 1.79x fewer generations, then 2.09x and 1.44x better result on the same budget one expensive run. thousands of free ones. the model did not get smarter. it got a memory it could rehearse in.

NO1ennn

32,776 просмотров • 18 дней назад

this is f*cking beyond comprehension. Google engineers just shipped the entire agent lifecycle in one release: build, scale, govern. and every piece answers a specific way agents die in production > context layers (Static, Turn, User, Cache): you decide what the model carries between turns, so token spend stops being a mystery > a self heal plugin: the agent notices a tool call failed and retries it a different way instead of dying mid run > adk deploy: one command from your laptop to the managed runtime, no packaging, no infra ticket > Go joins Python and Java, with its own A2A SDK then the part nobody builds for themselves: > a dashboard on token consumption, latency, error rates and tool calls: the four things that actually kill an agent > a traces tab that opens the real sequence of actions the agent took, step by step > a playground wired to the deployed agent, past sessions included, so debugging is not a redeploy loop > an Evaluation Layer with a User Simulator, because you cannot unit test a non deterministic system and the part that decides whether it ever ships: > agents get native identities as first class IAM principals: least privilege applies to them like it does to people > Model Armor screens prompt injection, tool calls and responses, inline for Gemini or over REST > Security Command Center inventories every agentic asset and flags data exfiltration by an agent ADK is already at 7 million downloads. the runtime has a free tier, and express mode runs off a Gmail address. the prototype was never the hard part.

NO1ennn

24,801 просмотров • 19 дней назад

sorry, they just did WHAT someone gave a machine one disease name, the leading cause of blindness in the developed world with 1.5 million americans already in its path, and it came back pointing at a drug that has sat in pharmacies for years under a different label: 551 papers read in 30 minutes against the 294 hours a human would have needed, and the loop that did it is public on GitHub most agent setups answer one question at a time, so the ceiling on the work is the quality of the question you happened to think of this one was handed a single question and wrote the second one itself. turns out that follow-up is where the real find was: a target called ABCA1, upregulated threefold, in an experiment no human ordered i read the whole paper looking for the trick, and the trick is structural. that is the second question, and it is the gap between an assistant and a factory: - hand the loop a field rather than a task: it was given a disease, and choosing the mechanism was part of its job - make it rank before it spends: 151 papers in, ten candidate mechanisms out, scored against each other before anything touched a bench - split reading from judging, so the agent that forms the theory is a different agent from the one grading it - close every cycle on physical reality: the verdict was an experiment, and another model's opinion was never allowed to stand in for one - feed each result back as the next question rather than a log line, which is the step almost nobody builds - search what already passed inspection first: the winner was an approved compound with a safety file already on record - write down what the round learned before opening the next one, so round two starts where round one stopped my read, and i think it is the uncomfortable one: reading was the entire bottleneck in that field, and everybody spent the decade optimising the writing. people ran every physical experiment here, the analysis agent needs a domain expert writing its prompts, and the authors decline to call this the leap it resembles. the thinking got replaced, and the hands did not so the question i cannot answer for my own setup: which step of your loop still stops dead until you sit down and type something bookmark this one. the four parts that turn one model into a line that runs like this, the queue, the rooms, the write permissions and the gate, are built file by file in the piece below ↓

Argona

32,744 просмотров • 1 месяц назад

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.

elvis

11,303 просмотров • 1 месяц назад