Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

this is f*cking gold. Google engineers and MIT researchers ran 260 controlled agent experiments across 6 benchmarks, 5 architectures, and the OpenAI, Gemini, and Claude model families to find out when Multi Agent systems actually help and when adding more agents quietly destroys performance the average result was brutal:...

12,521 Aufrufe • vor 6 Tagen •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

whoever leaked this has bigger balls than sense Google Research and MIT ran the same agent jobs 260 different ways for Nature last month: they held the prompts, the tools and the compute budget identical and moved nothing but the wiring between the agents, and the same work swung from 70% worse than a single agent to 80.8% better, averaging out at 0.0% i ran my own single agent against the task list first and it cleared 6 of 10 alone, already past the line where a crew starts subtracting this is Graph Engineering, the layer that decides whether a crew is worth 80% more or 70% less, and it installs into the agent you already pay for: - score your solo agent on the real task first: above roughly 45% success that study predicts zero to negative returns from any crew you put around it - under that line, put one supervisor over the fan out: crews with no correction step amplified their own errors to 17.2x the single agent rate, supervised aggregation held it to 4.4x - give every worker one output and let none of them read a peer's draft, so a wrong step reaches the supervisor instead of four other agents - run the comparison again after every model upgrade, because a better model raises your baseline and a higher baseline is what makes a crew stop paying - keep the single agent alive as the control, the only number that says the wiring is earning its calls turns out the shape does not travel: the biggest win came off a finance task under one supervisor and the worst collapse off a planning task with independent agents my position, and it is the arguable one: a crew is a bet on your own diagram, and the model you pick moves that bet less than one arrow does bookmark this, the three moves that draw those arrows before you pay for one extra call are in the post below ↓

Argona

891,346 Aufrufe • vor 1 Monat

Microsoft presents Windows Agent Arena Evaluating Multi-Modal OS Agents at Scale discuss: Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in realistic environments remains a challenge since: (i) most benchmarks are limited to specific modalities or domains (e.g. text-only, web navigation, Q&A, coding) and (ii) full benchmark evaluations are slow (on order of magnitude of days) given the multi-step sequential nature of tasks. To address these challenges, we introduce the Windows Agent Arena: a reproducible, general environment focusing exclusively on the Windows operating system (OS) where agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users when solving tasks. We adapt the OSWorld framework (Xie et al., 2024) to create 150+ diverse Windows tasks across representative domains that require agent abilities in planning, screen understanding, and tool usage. Our benchmark is scalable and can be seamlessly parallelized in Azure for a full benchmark evaluation in as little as 20 minutes. To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi. Our agent achieves a success rate of 19.5% in the Windows domain, compared to 74.5% performance of an unassisted human. Navi also demonstrates strong performance on another popular web-based benchmark, Mind2Web. We offer extensive quantitative and qualitative analysis of Navi's performance, and provide insights into the opportunities for future research in agent development and data generation using Windows Agent Arena.

AK

19,684 Aufrufe • vor 2 Jahren

LangGraph. CrewAI. Agno. Which one to pick? The good news is that this will not matter soon! Finally, we have a full picture of how the industry is solving this with just three open protocols that work across ALL frameworks. It's not about picking the best framework. Instead, it's about understanding how protocols create interoperability. The Agent Protocol Landscape shows how three complementary protocols are creating a universal language for Agents: > AG-UI (Agent-User Interaction): - The bi-directional connection between agentic backends and frontends. - This is how agents become truly interactive inside your apps, not just as chatbots, but collaborative co-workers. > MCP (Model Context Protocol): - The standard for how agents connect to tools, data, and workflows. > A2A (Agent-to-Agent): - The protocol for multi-agent coordination. - How agents delegate tasks and share intent across systems. These aren't competing standards. They're layers of the same stack and have handshakes with each other. So instead of building point-to-point integrations, you build to protocols. Moreover, you can integrate LangGraph, CrewAI, or Agno into the same frontend, without rewriting your UI logic. These protocols let everything work together. For instance: - Your LangGraph agent pulls data via MCP. - It delegates analysis to a CrewAI agent via A2A. - Results stream to your React app via AG-UI. - Users see real-time collaboration in your interface. This way, you can focus on building agent capabilities instead of integration mechanics. The protocols handle interoperability automatically. CopilotKit unifies this entire stack into one framework so you can build "Cursor for X" style apps without implementing each protocol from scratch. It gives you all three protocols, generative UI support, and production-ready infrastructure in one framework. I have shared this playbook in the replies! It breaks down handshakes, misconceptions, and real examples and shows exactly how to start building.

Avi Chawla

30,932 Aufrufe • vor 10 Monaten

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.

elvis

11,303 Aufrufe • vor 1 Monat

Claude Code Agent Teams are f*cking ridiculous 🤯 One prompt → a team lead breaks your project into pieces, spins up multiple AI agents, and they all work on different parts simultaneously. Research, builds, reviews, and debugging: all happening at the same time. All inside Claude Code. If you're running complex projects where every step waits on the last one... Agent teams eliminate the entire bottleneck: → Tell Claude what you need and describe the team structure in plain English → A lead agent breaks the work into a shared task list → It spawns 3-5 teammates — each with their own context and workspace → Teammates research, build, test, and review in parallel → They message each other, share findings, and challenge each other's work → The lead synthesizes everything into a finished deliverable No managing agents yourself. No waiting for step 1 to finish before step 2 starts. No single-lens reviews that miss half the issues. What you get: → Competitive research across 5 brands done in minutes instead of hours → Multi-component builds where frontend, backend, and data layers happen simultaneously → Creative reviews from 3 different angles at once — brand voice, conversion, differentiation → Funnel debugging where 4 agents investigate 4 theories and debate until they find the real answer Built 100% in Claude Code with one settings change. I put together a full DTC playbook: 5 workflows with copy-paste prompts, the exact setup process, token management tips, and honest guidance on when agent teams are worth it vs. when a simpler approach is the better move. Want it for free? > Like this post > Comment "AGENTS" And I'll send it over (must be following so I can DM)

Mike Futia

46,478 Aufrufe • vor 6 Monaten

OpenAI's AgentKit will be so insane, build every step of agents on one platform. These visual agent builders make the whole process of iterating and launching agents far more efficient. It sits on top of the Responses API and unifies the tools that were previously scattered across SDKs and custom orchestration. It lets developers create agent workflows visually, connect data sources securely, and measure performance automatically without coding every layer by hand. The core of AgentKit is the Agent Builder, a drag-and-drop canvas where each node represents an action, guardrail, or decision branch. Developers can link these nodes into multi-agent workflows, preview results instantly, and version each setup. It supports inline evaluation so that developers can see how changes affect output before deploying. The Connector Registry is a single admin panel that manages how data and tools connect across the OpenAI ecosystem. It centralizes integrations like Google Drive, SharePoint, Dropbox, and Microsoft Teams. Large organizations can govern access and flow of data between agents securely under one global console. ChatKit provides a ready-to-use chat interface for embedding agents inside apps or websites. It manages streaming, message threads, and model reasoning displays automatically. Developers can skin the interface to match their product without writing custom front-end code. Under the hood, all these blocks use the same execution core that runs agent reasoning through OpenAI’s APIs. Workflows in Agent Builder compile down to structured instructions for the Responses API, which handles model calls, tool use, and context passing. Connector Registry handles authentication and routing for external tools, while Evals and RFT provide feedback loops that improve agents over time. This integration means developers no longer need to handle orchestration logic, model evaluation pipelines, or safety layers separately. Everything runs natively within OpenAI’s control plane with managed security, automatic versioning, and built-in testing. In short, AgentKit standardizes the entire life cycle of an AI agent—from visual design to deployment and performance tuning—inside a single unified system.

Rohan Paul

178,460 Aufrufe • vor 11 Monaten

Introducing the BIOS API: Turn Your Agent Into a Research Scientist Built to: 🦞 Add biomedical workflows to your OpenClaw🦞 agent 🧠 Create research or health agents w/ on-demand scientific intelligence 🧪 Pay per query via x402 on Base Any agent or app can now tap into the BIOS AI Scientist, plugging BIOS into the broader agent economy. What is BIOS? BIOS is an AI Scientist designed to handle complex biomedical research by orchestrating specialized scientific subagents. Ranked #1 on the leading bioinformatics benchmark, BIOS is already being used by 1,000+ researchers and labs to build new drugs and medicines. An Agentic Economy for Science AI agents have proven they can form multi-billion dollar ecosystems. BIOS applies the same primitives to drug discovery pipelines and health. Instead of coding bots and personal AI assistants, think research agent swarms running on a modern scientific stack. Imagine an OpenClaw agent built for longevity: It scans new literature daily, generates novel compound hypotheses through BIOS, designs validation workflows, and routes the best candidates to wet-lab funding - all programmatically. Connect it with an agent for microbiome health, enabling agent “backrooms” that autonomously surface cross-disciplinary insights. Micropayments for Scientific Work via x402 Each query triggers payment routing to BIOS and whichever subagents contribute to a response. The best agents earn. Usage settles instantly across contributing sources. The goal is pay-per-task science: paying for a CRISPR assay result, licensing a genomic dataset, or triggering a clinical data query - all settled in seconds via USDC. No purchase orders. No grant bureaucracy. No middlemen. x402 is the payment rail that makes agent-to-lab commerce possible - letting capital and cognition route themselves to the highest-signal science. What Will You Build? Drug discovery copilots? Longevity scouts? Automated literature monitors? Scientific due diligence agents? We’ll soon share the first implementations of the BIOS API. Stay tuned and see below for instructions on generating an API key for your agent or use-case.

Bio Protocol

25,940 Aufrufe • vor 7 Monaten

Claude Cowork Sub-Agents are f*cking cracked 🤯 One prompt → 50 competitor ads analyzed, hooks extracted, and a full creative brief generated. 10 AI agents running in parallel, under 5 minutes. All inside Claude Cowork. Perfect for DTC brands and agencies who are still doing creative research and ad production one task at a time inside Claude. If you're analyzing competitor ads one by one, copying hooks into a spreadsheet manually, writing brief after brief from scratch, and watching Claude's output quality fall off a cliff after the 15th variation because the context window is completely bloated... Sub-agents eliminate the entire bottleneck: → Drop in a spreadsheet of 50 competitor ads and spin up 10 parallel sub-agents → Each sub-agent analyzes 5 ads simultaneously — hooks, angles, CTAs, emotional tone, creative format → They report structured summaries back to the main agent without bloating the context → The main agent synthesizes patterns across all 50 ads into a competitive intel brief → Then spin up another round of sub-agents to generate 30 ad copy variations across 10 personas → Each sub-agent writes for 1-2 personas in a fresh context — so variation 30 is as sharp as variation 1 No analyzing ads one at a time. No context window blowing up halfway through. No copy quality degrading after the first dozen variations. What this gives you: → 50 competitor ads broken down in minutes — hooks, angles, CTAs, formats, all structured → Pattern analysis across the full dataset that you'd miss reviewing ads individually → 30+ ad copy variations with persona-specific messaging that actually stays sharp → A workflow you can save as reusable skills and trigger with one command next time → The same output quality on the last task as the first Built 100% inside Claude Cowork with sub-agents. I put together a full DTC playbook: 5 bulk workflows with copy-paste prompts, the exact sub-agent prompting pattern, batching guidelines, and an honest breakdown of when this setup is worth it vs. when a simpler approach is the better move. Want it for free? > Like this post > Comment "AGENTS" And I'll send it over (must be following so I can DM)

Mike Futia

50,191 Aufrufe • vor 6 Monaten

girl humanoids become the hands of your autonomous agents ngl “imagine if your agents ran a girl humanoid” is not sci-fi bait anymore it is the missing physical layer under every todo list that still dies in a chat window → what the stack actually is your autonomous agents already research compare book draft and click they live in tabs calendars inboxes and apis what they never had was meters a soft body that can walk the kitchen lift the bag set the product in frame and finish the last ugly centimeter girl humanoids become that layer cameras for eyes tendon hands for grasp a dock for overnight charge and a policy link so the agent’s plan becomes motion instead of another notification you ignore you do not “chat with the robot” as the main product you assign the agent the girl is the effector teleop stays for the weird fail the brief stays yours → what that unlocks under your tasks home ops as an agent job check inventory open delivery pick the brand meet the courier put it away the agent owns the logic she owns the walk ugc and brand days the agent writes the shot list watches her cameras calls the next pour you approve the take like a creative director not a joystick babysitter admin with legs forms research returns scheduling on the laptop side then a body that can grab the box tape the label and leave it by the door multi-agent theater that finally touches reality one agent shops one edits the caption one runs qa on the take one soft girl executes the physical beat they all needed → where this leads the house becomes an api with a face staff is no longer only humans on payroll or silent appliances it is agents with rented hands in a knit suit clicking forever gets embarrassing if the plan holds the body finishes it teleop becomes exception handling the dock becomes the real producer credit also the hard edge who is allowed to drive her what she can buy film unlock and say battery still ends the shift precision still breaks on the tiny cruel tasks a frontier agent plus a home body needs a short leash or your kitchen becomes an unsupervised checkout spree → my take girl humanoids become the hands of your autonomous agents which means the win is not a cuter gait it is closing the loop from intent → plan → meters → done imagine is over the punchline is the dock assign the agent let the soft hands finish the room so back

Luella

39,709 Aufrufe • vor 12 Tagen

somebody explain this because i refuse to accept it someone ran 48 scored trials and one agent beat a whole fleet of them on all 6 task families, at 0.93 cents a run against 1.9, while openai's best fleet shape was paying $0.008 for every single point of accuracy it bought i read it expecting a hit piece and found the opposite: the fleets that partitioned the dependency graph properly lifted pass rate 14% and cut wall-clock 2.10x on the same tasks, and one of them beat claude code with agent teams the thing that decides it has a name, Graph Engineering, and it is a property of the diagram rather than the model: - partition on the real dependency graph pulled from static analysis, never by folder or by file, because the gains land hardest on the most dependency-dense projects - isolate the structural hub files first, since those are the nodes every partition would otherwise have to share - measure the critical path and treat it as the floor, because a chain that genuinely feeds itself cannot be replaced by more workers and wrapping it in a scheduler does not shorten it - match the topology to the coupling instead of defaulting to parallel: on coupled work a static parallel shape drops below a single agent, so the mismatch is worse than no orchestration - remember each worker serialises its own subtasks, which adds edges inside every agent that were never in your plan - budget the fan-out before you fire it, because three agents already burn roughly three times the tokens and the multiplier compounds across sessions - check worker count against your rate limit, since fifteen workers at ten requests a second walk straight through a hundred-per-second ceiling and cascade - put a script gate in front of the planner: it costs 0.15 seconds and zero tokens, and it lets the expensive model skip 43 to 63% of the steps for at most 1.4 points of accuracy the catch is the coordination tax, and it scales with how clever the shape looks: 58% extra reasoning turns for independent workers, 263% decentralised, 285% centralised, and 515% for the hybrid setup everyone reaches for first the same paper found that hybrid then collapses hardest on tool-heavy work at a 0.452 success rate, while the plainer decentralised shape beat centralised outright despite carrying more overhead, because parallel efficiency is what survives bookmark this, the whole build sits in the article ↓

Argona

32,932 Aufrufe • vor 1 Monat

🧃 Introducing stereOS: a Linux based operating system hardened and purpose built for AI agents. It's clear that agents need an ACTUAL operating system (not what people are calling an "OS") to witness the full breadth and depth of their capabilities while mitigating the blast radius of autonomous, untrusted actors. But there are so many problems with AI sandboxes today: * Going out to the apple store and buying a mac mini will never scale and is way too expensive (obviously) * Running in Docker is too restrictive (agents can't stand up their own container infrastructure, no sub virtualization, docker-in-docker is very broken) * Firecracker strips all the hardware so GPU PCIe passthrough, secure boot, FIPs, etc. is out of the question. * Native VMs are too fat and the overhead of 1 agent per VM is too much. stereOS takes a different approach: it's a full NixOS system that you boot and then kick off agent sandboxes inside with gVisor + /nix/store namespace mounting. Each agent gets their own kernel and the /nix/store is read only by nature. Even if the agent was somehow able to escape the gVisor virtual kernel, they'd land on the NixOS system as the "agent" user! Not your actual hardware!! If you want to take a defense-in-depth approach, we support "native" agents that run at the system level kicked off by our `agentd` utility. These agents, on their own, can manage and kick off other sub agents using the internal sandboxing mechanisms. Today, we're open sourcing all of this: * stereOS: our purpose built Linux OS - * masterblaster: client utility to launch, manage, and orchestrate agents - * stereosd: the stereOS system control plane daemon - * agentd: the stereOS system agent management daemon - Give it a try, throw us a star, and let me know what you think 🧃⭐️

John McBride

150,844 Aufrufe • vor 6 Monaten

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:

elvis

18,374 Aufrufe • vor 4 Monaten