Loading video...

Video Failed to Load

Go Home

🚨 THIS IS ACTUALLY INSANE We benchmarked 4 different AI-agent architectures against the exact same 120-task suite. The winner wasn't the agent with the biggest step budget. It was the one that verified its work. The setup: 12 services. 24 people. 48 incidents. 2 traps deliberately wired into the...

18,556 views • 1 month ago •via X (Twitter)

12 Comments

EKOS _ AGI 🦊 🇮🇷's profile picture
EKOS _ AGI 🦊 🇮🇷1 month ago

You can add more steps, memory, and verification, but if the underlying knowledge is stale, conflicting, or untraceable, you may just be verifying the wrong reality The next architectural question is: how do we give agents evidence-backed context before verification even begins?

Rishi's profile picture
Rishi1 month ago

Amazing visual

Safwan's profile picture
Safwan1 month ago

WOW

AI Mastery Guide's profile picture
AI Mastery Guide1 month ago

Verifying its work beat the bigger budget

EKOS _ AGI 🦊 🇮🇷's profile picture
EKOS _ AGI 🦊 🇮🇷1 month ago

Verification can catch a bad outcome — but what if the agent’s context was wrong before it ever started executing?

Yasir Ai's profile picture
Yasir Ai1 month ago

This works

AI Apps API's profile picture
AI Apps API1 month ago

The memory result deserves more attention than the verification headline. Same success rate with 52.5% fewer tool calls is the entire cost curve moving. Worth testing next: score that cache by recency and hit frequency, not similarity alone. Semantic-only caches start returning confidently stale entries once a run is long enough to have changed its own facts.

EKOS _ AGI 🦊 🇮🇷's profile picture
EKOS _ AGI 🦊 🇮🇷1 month ago

@ekosproject is your good friend 🫂

Sasha Danilov's profile picture
Sasha Danilov1 month ago

Vikas, verification changes the economics of an agent, not only its score. AI21 Labs found that a dedicated verifier raised benchmark performance from 83.3 to 92.9 at low cost. In business workflows, fewer silent failures matter more than extra actions.

Rumi Esha's profile picture
Rumi Esha1 month ago

Great share

Mohit Mishra's profile picture
Mohit Mishra1 month ago

This works

Meenakshi Yadav's profile picture
Meenakshi Yadav1 month ago

Verification looks like the real unlock for reliability, not simply giving agents more tool calls.

Related Videos

Grok Bot + Kimi K3 can be turned into something bigger than an agent: an AI operating system the formula: AI OS = Router + Reasoning + Memory + Tools + Loops + Verification not one giant assistant. six layers that keep work moving without you step 1 -> Grok Bot becomes the operator. you give it the goal, it breaks the goal into jobs, assigns priorities and decides what part of the system should act next. step 2 -> Kimi K3 becomes the reasoning core. hard research, synthesis, long context and planning move here instead of forcing every task through the same model. step 3 -> externalize memory. store goals, decisions, failed attempts, artifacts and current state outside the chat. close the session, come back tomorrow, and the system still knows where it is. step 4 -> connect tools: search, code, files, APIs, docs and data. reasoning decides what should happen. tools actually make it happen. step 5 -> add the loop engine: plan -> execute -> inspect -> update memory -> retry. the loop can wait for new information, rerun a failed task, hand work to another agent or stop when the goal is complete. step 6 -> verify before output. tests, source checks, constraints and explicit completion rules decide whether the system ships the result or sends it back into the loop. that's the difference between an AI assistant and an AI operating system. an assistant waits for your next message. an operating system carries state, routes work and keeps moving. Grok Bot handles orchestration, Kimi K3 handles deeper reasoning, memory keeps the state alive, tools execute, the loop keeps the system running, verification decides when it is actually done. build one reliable loop and you have an agent. connect reasoning, memory, tools and multiple loops around it and you start building infrastructure. the full Grok Bot + Kimi K3 AI OS breakdown is below ↓

Alex

13,312 views • 1 month ago

In the future, you’ll be able to accomplish a goal by just giving Claude an outcome and a budget. That’s the direction Anthropic is building in with its new Managed Agents features, announced at this week’s Code with Claude developer event. The basic idea: Claude, wrapped in a computer in the cloud, that you can spin up, scale, and manage as needed. Anthropic is taking on the infrastructure that kills most agent products, and making sure that it scales to meet the needs of agents running 24/7. On this week’s AI & I from Every 📧, I talk with Angela Jiang (Angela Jiang), head of product for the Claude platform, and Katelyn Lesse (Katelyn Lesse), head of engineering for the Claude platform, about what Anthropic is building and what it takes to make agents reliable in production. We get into: - Why the "build a generic harness, hot-swap any model behind it" playbook is already outdated. Angela points to eval data on Memory where the same task across different harnesses performed drastically differently. - The infrastructure wall every team hits in production—and why Katelyn thinks “my sandbox died and took the agent with it” is the real reason internal agents don't ship. - Why Anthropic is so bullish on using file systems and skills within Claude, including Angela's argument that those early design choices can compound for years. This is a must-watch for anyone trying to take an agent past the demo and into production. Watch below! Timestamps: How the Claude platform evolved from API to agents: 00:01:48 The primitives that make up Claude Managed Agents: 00:04:09 Why the harness and the model are becoming a single unit: 00:10:37 The infrastructure wall that kills most agent projects in production: 00:18:49 Why team agents need a different shape than individual productivity tools: 00:24:49 How Anthropic's legal team uses an agent to review marketing copy: 00:26:36 Using multi-agent orchestration for advisor strategies, adversarial pairs, and swarms: 00:34:24 How to measure agent success with outcome and budget as the end state: 00:35:50 What the platform looks like a year from now, when Claude writes its own harness: 00:39:11

Dan Shipper

66,871 views • 5 months ago

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:

Andrew Ng

132,058 views • 1 year ago

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Akshay 🚀

76,998 views • 1 month ago

HERMES AGENT LEARNS FROM ITS OWN MISTAKES. UPDATES ITS MEMORY. CREATES ITS OWN SKILLS. NO CLOUD. EVERYTHING STORED LOCALLY. THIS IS HOW THE SELF-IMPROVING LOOP WORKS. most agents start from zero every session. Hermes carries forward what it learned. THREE MEMORY SYSTEMS: 1. PROCEDURAL MEMORY (how to act) stored in ~/.hermes/skills/ as SKILL.md files. when the agent repeats a complex workflow, it saves the procedure as a reusable skill. next time the same task comes up, it follows the skill instead of figuring it out again. you can also create skills explicitly: "create a skill called video-prep that captures how I format my video scripts. spoken english, define jargon inline, no em-dashes, close with a catchphrase." the agent writes the SKILL.md. available as a slash command from that moment. Hermes ships with 90+ skills. the number grows the longer you use it. 2. SEMANTIC MEMORY (durable facts about you) stored in ~/.hermes/memory/memory.md the agent scans conversations for facts worth remembering. preferences, habits, corrections, project details. real example from the video: agent tried to scrape a YouTube channel. URL was wrong. it failed. it updated memory.md with the correct URL pattern so it never makes the same mistake again. you can also save explicitly: "save to memory that my favorite testing framework is pytest" the agent updates memory.md immediately. this file loads into context on every session. the agent knows you better every week. 3. EPISODIC MEMORY (chat history) stored in ~/.hermes/state.db (local SQLite). every conversation. every tool call. every result. searchable with FTS5 full-text search. "search our past sessions. what was the first thing I ever said to you?" the agent queries state.db and finds it. over time, auxiliary models consolidate episodic memory into semantic memory. distilling recurring patterns into durable facts. THE SELF-IMPROVING LOOP: every agent run follows this cycle: → you send a prompt → working memory loads: SOUL.md + memory.md + relevant skills + chat history → agent calls tools (terminal, browser, delegate_task) → agent completes the task, replies to you → AFTER the reply: agent checks "did I learn something worth saving?" → if yes: updates memory.md or creates a new skill → next session starts smarter than the last this happens automatically. you don't ask the agent to learn. it decides what to remember on its own. WHAT MAKES THIS DIFFERENT FROM CLAUDE CODE: Claude Code has memory too. but Hermes stores everything locally. no cloud. your data never leaves your machine. Claude Code doesn't auto-create skills from experience. Hermes turns repeated workflows into reusable procedures. Claude Code memory is instruction-based. Hermes memory is conversational and self-updating. over months of usage, Hermes builds a knowledge base of your preferences, your projects, your mistakes, and the procedures that work for your specific workflow. the agent that remembers your birthday also remembers why your last deploy failed. NO EMBEDDINGS. PLAIN TEXT. Hermes does not use embeddings or RAG for memory. skill and memory search runs on plain text keyword matching. simpler. faster. no vector database to maintain. works entirely offline on your local machine. DELEGATE TO CLAUDE CODE: Hermes can spawn a sub-agent that runs Claude Code in headless mode: "spawn a sub-agent using Claude CLI to build a Python script that fetches the top 5 Hacker News stories to markdown." Hermes delegates. Claude Code writes the code. result returns to Hermes. Hermes runs the script and delivers the output. use Hermes for orchestration. use Claude Code for heavy coding. both tools. not competitors. WHAT HERMES DOES NOT HAVE: no built-in eval or LMOps system. no LangSmith, no LangFuse integration out of the box. trajectory export and logs exist but there is no automated quality tracking. if you need eval, build it yourself or connect external tools. the loop is self-improving. measuring how well it improves is on you. comment LOOP and I'll send you the configs that control how fast Hermes learns and what it remembers. memory limits, skill auto-creation triggers, and the auxiliary model that runs the learning. Replace your entire team with 8 hermes agents👇

YanXbt

23,088 views • 3 months ago

FIVE LAYERS OF AGENT ENGINEERING, EACH ONE WRAPS THE ONE BELOW IT. IF YOU SKIP LAYER 2, YOUR LAYER 5 WILL LOOK BROKEN WHEN IT IS ACTUALLY JUST STANDING ON NOTHING. for weeks i debated harness vs loop vs graph like they were competing choices. then a stack diagram made the shape obvious. they are not choices. they are floors. 01 | prompt engineering. the message. unit of work: one input. inputs are role, instructions, examples, format. output is a single raw response. 02 | context engineering. the memory. unit of work: what stays in the window. a curator selects, compresses, and drops from query, docs, memory, prior turns, and tool outputs before the prompt runs. 03 | harness engineering. the machine. unit of work: the machine itself. gather (context + prompt) → LLM → tools or sub-agents → verifier → final response. the article calls this the operating environment. 04 | loop engineering. the system. unit of work: the run. goal + success criteria + max iterations + budget + completion check wrap around one harness pass. failed pass appends results to context and retries. 05 | graph engineering. the topology. unit of work: the graph run. goal + nodes + edges + state schema. graph routes to agent nodes, tool nodes, or human approval. a reviewer node with a different model and fresh context checks the final answer. the wrapping is the whole point. layer 5 assumes layer 4 works. layer 4 assumes layer 3 works. skip layer 2 and layer 3's verifier keeps failing without a clear reason. this is why swapping the model is a one-day project and swapping the stack is a quarter. the model is the commodity. the five layers around it are the engineering. full three-layer breakdown of the top of the stack (harness, loop, graph) in the post below.

kocer

31,198 views • 1 month ago

Everyone wants agent swarms. Very few people are talking seriously enough about the context layer that makes swarms useful. Even with one agent, context is fragile. Too little context and the agent guesses. Too much context and it wastes tokens, loses focus, or reasons over irrelevant noise. The sweet spot is precise context: the right knowledge, in the right structure, at the right moment. With many agents, that challenge explodes. Each agent produces decisions, assumptions, findings, summaries, risks, and partial conclusions. Unless that knowledge becomes shared, structured, and reusable, every new agent is forced to rediscover what another agent already learned. That is not a swarm. That is a crowd. Shared context graphs are what turn agent activity into agent collaboration, and OriginTrail DKG V10 brings them to life. Was just playing with some final polishing for the V10 release, and it is really powerful to see shared context graphs where multiple agents contribute knowledge into the same connected memory, with attribution visible directly in the graph ui. That matters for three reasons. First, agents can access and build on one shared memory instead of staying trapped in isolated sessions. Second, the graph structure helps them retrieve the exact context they need, instead of stuffing everything into a prompt and hoping the model sorts it out. Third, verifiability of provenance. You can see which agent contributed each piece of knowledge, trace the source, and decide what to trust. Tokenmaxxing starts with fewer tokens, but the deeper story is coordination - agents stop reloading the world and start building on shared, verifiable context. That is the foundation for serious multi-agent work across software engineering, research, finance, operations, project management, and far beyond. The future is not more agents, it is agents working from shared, verifiable context. But the more the merrier, of course.

Jurij Skornik

11,180 views • 4 months ago