Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Claude Code vs Codex vs Pi: which coding agent wins? Melissa Pan, PhD candidate at UC Berkeley’s Sky Computing Lab and previous Arena.ai intern, explored the hidden “harness tax”: how the system surrounding an AI model affects its cost and performance. She reports three surprising findings. One: harness choice... show more

Arena.ai

233,336 subscribers

85,347 views • 11 days ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

22 Comments

Eric Stevens11 days ago

@melissapan Swap the model and barely anything changes. Rebuild the scaffolding around it and suddenly it's a different tool.

Curious Nori | AI · Food · Travel11 days ago

@melissapan Well said.

VastPlan11 days ago

@melissapan Arena 对照 Claude Code、Codex 与 Pi，指出外围 harness 带来的成本差往往大于准确率差。代理任务的费用不只取决于模型单价，工具编排、重试与上下文装填会显著改写单次账单；选栈时应把 harness 与模型一并计入。

tootooc11 days ago

@melissapan 同模型换壳，账单和成功率一起飘——这句比榜单管用。一人公司选型别只盯 benchmark：环路怎么验、失败怎么停、状态写在哪。壳可以换；「过/不过」别寄存在某一个壳的默认里。

owais | building 🏴‍☠️11 days ago

@melissapan Harness mattering more for cost than accuracy makes sense when so much of the bill is context re-sent every turn. Would love to see cost per solved task broken out by harness.

Rach11 days ago

@melissapan @trycodegraff! as well!! it’s a self evolving harness haha

安叫兽|Bird🕊️ 🔶 BNB11 days ago

@melissapan 原来测试套件才是偷偷抬成本的那只手

VB11 days ago

@melissapan harness tax is the real scoreboard. same model, totally different bill depending on how messy the loop around it is

RAZA | AI EXPLORER11 days ago

@melissapan The harness tax is a fascinating angle. Really useful analysis for building efficient coding agents.

Daniel11 days ago

@melissapan can someone give me the tldr 🤓

Automater11 days ago

@melissapan The 'which coding agent wins' answer is usually whichever harness stopped fighting the model. Measure tool-loop stability under load, not demo speed. #ClaudeCode

Artzy11 days ago

@melissapan The harness tax is an often overlooked performance killer

ethereagle · building11 days ago

@melissapan if harness moves cost more than accuracy, you're ranking the wrapper as much as the model. was that Claude Code vs Codex vs Pi on the same model, or did each keep its default?

Asad Ullah 🇵🇸11 days ago

@melissapan I think pi cost effective and powerful coding agent right now

Raven11 days ago

@melissapan the model gets graded, the harness quietly invoices everyone

Jeremy Fiance11 days ago

@melissapan This is awesome/helpful Melissa & Arena!

Hunter Bown11 days ago

@melissapan would love to see how codewhale performs!

Silent · Dev Tools11 days ago

Harness tax decides the winner more than the model card. Tonight: time-to-first-kept-diff on the same bug across Claude Code / Codex / your third agent, with identical AGENTS.md. Whichever burns fewer tokens per kept line for a week becomes the default lane — swap the rest to overflow only.

Fajar M Reza11 days ago

@melissapan Agent benchmarks need matched tasks and budgets; harnesses can dominate outcomes.

IrradiatedFentOwl11 days ago

@melissapan Female woman PagMan

catman11 days ago

@melissapan Benchmark the whole harness on representative tasks, including tool calls and retries, before picking a model. If harness choice drives cost more than accuracy, optimize routing and stop conditions first.

Shahid🇵🇸11 days ago

@melissapan did she break down prompt cache hit rate per harness? I'd guess that's where most of the cost gap comes from, more than anything the harness does to accuracy

Related Videos

I'm sharing with you one of the best papers I've read on Harness Engineering it takes apart Claude Code (Opus 5.5), Codex, Gemini CLI and more, and boils all of them down to one line: agent = model + harness what you walk away with: > what a harness actually is > how today's coding agents are built under the hood > the patterns every one of them repeats > where the whole category is heading > their checklist for building your own the model gets the headlines. the harness decides whether the agent works loop and graph engineering on top of it - in my article below

I'm sharing with you one of the best papers I've read on Harness Engineering it takes apart Claude Code (Opus 5.5), Codex, Gemini CLI and more, and boils all of them down to one line: agent = model + harness what you walk away with: > what a harness actually is > how today's coding agents are built under the hood > the patterns every one of them repeats > where the whole category is heading > their checklist for building your own the model gets the headlines. the harness decides whether the agent works loop and graph engineering on top of it - in my article below

Mr. Buzzoni

73,455 views • 4 days ago

Jev founder, Diogo Almeida, just put out a PDF on building a Jev harness for coding agents the claim: 200x faster, 400x cheaper the model stopped being the bottleneck a while ago. the harness around it is where the cost and the speed actually live five decisions sit on every loop, and only one of them writes code hand this PDF and the article below to your Claude Code or Codex instance and let it rebuild its own setup your agent gets a better harness tonight, and monday looks different than it would have 👇

Jev founder, Diogo Almeida, just put out a PDF on building a Jev harness for coding agents the claim: 200x faster, 400x cheaper the model stopped being the bottleneck a while ago. the harness around it is where the cost and the speed actually live five decisions sit on every loop, and only one of them writes code hand this PDF and the article below to your Claude Code or Codex instance and let it rebuild its own setup your agent gets a better harness tonight, and monday looks different than it would have 👇

Hanako

37,887 views • 15 days ago

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness the promise: > 200x faster > 400x cheaper the model hasn't been the slow part for a while the speed and the cost sit in the harness wrapped around it • how to use it > drop this PDF and the article below into Claude Code or Codex > let it rebuild its own setup one evening of setup and your agent starts next week on a harness most teams still don't have 👇

this is pure f*cking gold for anyone running coding agents Jev founder Diogo Amogo wrote a PDF on building a Jev harness the promise: > 200x faster > 400x cheaper the model hasn't been the slow part for a while the speed and the cost sit in the harness wrapped around it • how to use it > drop this PDF and the article below into Claude Code or Codex > let it rebuild its own setup one evening of setup and your agent starts next week on a harness most teams still don't have 👇

Mr. Buzzoni

128,034 views • 14 days ago

This GitHub repo is gold for anyone running Claude Code/ Codex / Any coding agent at all. My usage limits almost last double since I started using this. An open source context layer that stops your agent from reading 40 files to answer one question. No API key, no billing, no rate limit tiers. Benchmarked against Claude Code baseline: 27x token efficiency, 36% lower cost, 89% fewer file reads, 49% fewer tool calls. Every number is reproducible, the harness is in the repo. 3.5k+ stars. 50k+ pip installs, 30+ contributors, works with Claude Code, Cursor and VS Code out of the box. pip install repowise and point it at your codebase.

This GitHub repo is gold for anyone running Claude Code/ Codex / Any coding agent at all. My usage limits almost last double since I started using this. An open source context layer that stops your agent from reading 40 files to answer one question. No API key, no billing, no rate limit tiers. Benchmarked against Claude Code baseline: 27x token efficiency, 36% lower cost, 89% fewer file reads, 49% fewer tool calls. Every number is reproducible, the harness is in the repo. 3.5k+ stars. 50k+ pip installs, 30+ contributors, works with Claude Code, Cursor and VS Code out of the box. pip install repowise and point it at your codebase.

Srishti

12,998 views • 3 months ago

Shopify CEO Tobi Lütke called Pi "the most interesting agent harness" - a tiny, open-source framework built by a single developer that powers OpenClaw and is changing how builders think about AI agents. In this clip, Dave Killeen (Field CPO at Pendo) breaks down why Pi matters, how it compares to Manus, and why the real magic in AI isn't the language model itself - it's the orchestration layer around it. Pi is a lightweight agent harness that lets developers build smarter AI experiences by leveraging models like Claude, GPT, and Gemini through better orchestration, not bigger models. If you're building AI products, coding agents, or exploring the new world of harness engineering, this is essential context. We discuss why Manus gave users a far better experience than talking directly to an LLM, and why open-source tools like Pi could be the future of agent development.

Shopify CEO Tobi Lütke called Pi "the most interesting agent harness" - a tiny, open-source framework built by a single developer that powers OpenClaw and is changing how builders think about AI agents. In this clip, Dave Killeen (Field CPO at Pendo) breaks down why Pi matters, how it compares to Manus, and why the real magic in AI isn't the language model itself - it's the orchestration layer around it. Pi is a lightweight agent harness that lets developers build smarter AI experiences by leveraging models like Claude, GPT, and Gemini through better orchestration, not bigger models. If you're building AI products, coding agents, or exploring the new world of harness engineering, this is essential context. We discuss why Manus gave users a far better experience than talking directly to an LLM, and why open-source tools like Pi could be the future of agent development.

Aakash Gupta

10,774 views • 29 days ago

An agent is three things: a harness, a model, and context. If you're serious about owning your intelligence, you probably want to own all three. LangChain founder Harrison Chase joined us at our Sequoia Capital Own Your Intelligence to talk about the piece that often gets the least attention: the harness. He offers a clear heuristic for when to build your own. The more out of distribution you are from what the models were trained on, the more you'll want to customize. And good technical content on how to actually measure performance with evals and langsmith. 00:00 Introduction 00:58 The three parts of an agent: harness, model, context 02:12 What a harness actually does 03:25 Customizing the core loop with middleware 04:41 Sandboxes, file systems, sub-agents, summarization 05:47 Cognitive architectures — and when you still need them 07:03 Build your own harness or use off the shelf? 08:24 In-distribution vs. out-of-distribution: the file-editing example 09:39 Why evals define what "good" means in an organization 11:04 Harbor: what an eval task actually looks like 12:11 Comparing harnesses and models on accuracy, latency, and cost 13:20 Why observability is underrated — it's usually the context 14:34 The data flywheel: traces → curation → experiments 15:42 Getting feedback through UX design and online evaluators 16:51 Demo: LangSmith Engine 19:23 Q&A: Running Engine on Engine, and "codex-ification" 20:44 Q&A: Will harnesses converge or diverge?

An agent is three things: a harness, a model, and context. If you're serious about owning your intelligence, you probably want to own all three. LangChain founder Harrison Chase joined us at our Sequoia Capital Own Your Intelligence to talk about the piece that often gets the least attention: the harness. He offers a clear heuristic for when to build your own. The more out of distribution you are from what the models were trained on, the more you'll want to customize. And good technical content on how to actually measure performance with evals and langsmith. 00:00 Introduction 00:58 The three parts of an agent: harness, model, context 02:12 What a harness actually does 03:25 Customizing the core loop with middleware 04:41 Sandboxes, file systems, sub-agents, summarization 05:47 Cognitive architectures — and when you still need them 07:03 Build your own harness or use off the shelf? 08:24 In-distribution vs. out-of-distribution: the file-editing example 09:39 Why evals define what "good" means in an organization 11:04 Harbor: what an eval task actually looks like 12:11 Comparing harnesses and models on accuracy, latency, and cost 13:20 Why observability is underrated — it's usually the context 14:34 The data flywheel: traces → curation → experiments 15:42 Getting feedback through UX design and online evaluators 16:51 Demo: LangSmith Engine 19:23 Q&A: Running Engine on Engine, and "codex-ification" 20:44 Q&A: Will harnesses converge or diverge?

Sonya Huang 🐥

77,519 views • 1 month ago

Introducing the context course: a free course on doing ML with agent context. You will learn how to train models, optimize inferences, and build datasets, all by defining harness context with`SKILLS.md`, Plugins, MCP, Subagents, and Hooks. The course includes: - Weekly live AMA on YouTube - Weekly practical projects for ML with context - Instructions in Pi, Codex, Claude, and Opencode - Tutorials and guides on fundamentals - Interactive Quizzes Learn to give AI agents the right knowledge, tools, and structure to actually get work done. Skills, MCP servers, plugins, multi-agent workflows, and building an agent from scratch. Join here:

Introducing the context course: a free course on doing ML with agent context. You will learn how to train models, optimize inferences, and build datasets, all by defining harness context with`SKILLS.md`, Plugins, MCP, Subagents, and Hooks. The course includes: - Weekly live AMA on YouTube - Weekly practical projects for ML with context - Instructions in Pi, Codex, Claude, and Opencode - Tutorials and guides on fundamentals - Interactive Quizzes Learn to give AI agents the right knowledge, tools, and structure to actually get work done. Skills, MCP servers, plugins, multi-agent workflows, and building an agent from scratch. Join here:

Ben Burtenshaw

16,905 views • 5 months ago

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for reasoning over extracting out complex information from complex docs. Each extracted field comes with a confidence score as well as a bounding box denoting where it came from. It does 10-20% better in accuracy than generalized coding agent harnesses (e.g. Claude Code Opus 4.8 and Codex GPT-5.6). Check out the video below for a demo. The mode is called LlamaExtract Agentic Plus. Learn more about our extraction benchmark and agentic plus mode here: If you have very complex document extraction needs, come check out LlamaParse!

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for reasoning over extracting out complex information from complex docs. Each extracted field comes with a confidence score as well as a bounding box denoting where it came from. It does 10-20% better in accuracy than generalized coding agent harnesses (e.g. Claude Code Opus 4.8 and Codex GPT-5.6). Check out the video below for a demo. The mode is called LlamaExtract Agentic Plus. Learn more about our extraction benchmark and agentic plus mode here: If you have very complex document extraction needs, come check out LlamaParse!

Jerry Liu

18,359 views • 1 month ago

this might be the most useful AI agent paper of 2026, and it's certified f*cking gold Wavestone AI Lab took apart Claude Code, Codex and 9 more agents every good one runs on the same 7-part harness agent = model + harness all 11 use the same 7 parts. only the size changes the 7 parts: > loop - from a plain while loop to fully replayable runs > LLM layer - one prompt template up to 29 provider profiles > tools - bash only up to 43 typed tools on demand > memory - full history up to notes kept across sessions > safety - a step limit up to rules + reviewer model + sandbox > orchestration - Aider skips sub-agents, others fork them > extensions - SKILL.md in 9 of 11, MCP in 8 it ends with a 90-line harness covering all 7 the cleanest starting point for your own agent the pages worth your time: > p.4 - the 7-part map > p.12 - 3 loop types, Claude Code and Codex as examples > p.53 - why nobody uses frameworks or embeddings > p.67 - 18 design rules > p.71 - the 90-line harness to copy

this might be the most useful AI agent paper of 2026, and it's certified f*cking gold Wavestone AI Lab took apart Claude Code, Codex and 9 more agents every good one runs on the same 7-part harness agent = model + harness all 11 use the same 7 parts. only the size changes the 7 parts: > loop - from a plain while loop to fully replayable runs > LLM layer - one prompt template up to 29 provider profiles > tools - bash only up to 43 typed tools on demand > memory - full history up to notes kept across sessions > safety - a step limit up to rules + reviewer model + sandbox > orchestration - Aider skips sub-agents, others fork them > extensions - SKILL.md in 9 of 11, MCP in 8 it ends with a 90-line harness covering all 7 the cleanest starting point for your own agent the pages worth your time: > p.4 - the 7-part map > p.12 - 3 loop types, Claude Code and Codex as examples > p.53 - why nobody uses frameworks or embeddings > p.67 - 18 design rules > p.71 - the 90-line harness to copy

Annatar.md

50,060 views • 5 days ago

Claude Code is a major (and accidental!) hit for Anthropic that surprised even its creator, Boris Cherny. Claude Code, an Agentic AI coding product that lives in the terminal. Most of the new code at Anthropic is created through it today. And in the last 5 months since it was launched publicly, Claude Code went from $0 to $400M in revenue run rate (as per The Information). 00:00 – Intro 01:15 – Did You Expect Claude Code’s Success? 04:22 – How Claude Code Works and Origins 08:05 – Command Line vs IDE: Why Start Claude Code in the Terminal? 11:31 – The Evolution of Programming: From Punch Cards to Agents 13:20 – Product Follows Model: Simple Interfaces and Fast Evolution 15:17 – Who Is Claude Code For? (Engineers, Designers, PMs & More) 17:46 – What Can Claude Code Actually Do? (Actions & Capabilities) 21:14 – Agentic Actions, Subagents, and Workflows 25:30 – Claude Code’s Awareness, Memory, and Knowledge Sharing 33:28 – Model Context Protocol (MCP) and Customization 35:30 – Safety, Human Oversight, and Enterprise Considerations 38:10 – UX/UI: Making Claude Code Useful and Enjoyable 40:44 – Pricing for Power Users and Subscription Models 43:36 – Real-World Use Cases: Debugging, Testing, and More 46:44 – How Does Claude Code Transform Onboarding? 49:36 – The Future of Coding: Agents, Teams, and Collaboration 54:11 – The AI Coding Wars: Competition & Ecosystem 57:27 – The Future of Coding as a Profession 58:41 – What’s Next for Claude Code

Claude Code is a major (and accidental!) hit for Anthropic that surprised even its creator, Boris Cherny. Claude Code, an Agentic AI coding product that lives in the terminal. Most of the new code at Anthropic is created through it today. And in the last 5 months since it was launched publicly, Claude Code went from $0 to $400M in revenue run rate (as per The Information). 00:00 – Intro 01:15 – Did You Expect Claude Code’s Success? 04:22 – How Claude Code Works and Origins 08:05 – Command Line vs IDE: Why Start Claude Code in the Terminal? 11:31 – The Evolution of Programming: From Punch Cards to Agents 13:20 – Product Follows Model: Simple Interfaces and Fast Evolution 15:17 – Who Is Claude Code For? (Engineers, Designers, PMs & More) 17:46 – What Can Claude Code Actually Do? (Actions & Capabilities) 21:14 – Agentic Actions, Subagents, and Workflows 25:30 – Claude Code’s Awareness, Memory, and Knowledge Sharing 33:28 – Model Context Protocol (MCP) and Customization 35:30 – Safety, Human Oversight, and Enterprise Considerations 38:10 – UX/UI: Making Claude Code Useful and Enjoyable 40:44 – Pricing for Power Users and Subscription Models 43:36 – Real-World Use Cases: Debugging, Testing, and More 46:44 – How Does Claude Code Transform Onboarding? 49:36 – The Future of Coding: Agents, Teams, and Collaboration 54:11 – The AI Coding Wars: Competition & Ecosystem 57:27 – The Future of Coding as a Profession 58:41 – What’s Next for Claude Code

Matt Turck

82,372 views • 1 year ago

We just crossed $10M in ARR at Chatbase! 🎉 🎉 And today, we're launching Chatbase as the full harness for customer-facing AI agents. Similar to how Claude code is a harness for coding agents, Chatbase is the harness for customer experience agents. That means we give the model the context, tools, workflows, guardrails, and human-in-the-loop systems to be the best ambassador for your brand. It's going beyond just solving issues and is giving your customers the best experiences across every channel. This is a milestone I have been thinking about and obsessed with since day 1, and I am super excited to bring my vision for customer facing agents to life with Chatbase. Thank you to every one of our customers and to the amazing Chatbase team for getting us here! Next stop: $100M ARR

Yasser

853,076 views • 4 months ago

Anthropic engineer: "Harness engineering is a really rare skill right now - and it's hard to know if you're even good at it." how he actually thinks about building the full system: • step 1 → the model. already smarter than we use it. "freeze development - you'd find new capabilities for another 6-12 months" • step 2 → the harness. delete the steering. "we stopped using plan mode - the model thinks correctly" • step 3 → the agents. workflows let Claude build its own harness per task and spin sub-agents. what didn't fit in one context window now runs across many • step 4 → the context. "RAG is an anti-pattern. use grep." memory via files, clean context window, prompt caching. that's the stack most people optimize step 1. he optimizes all four. Watch the full video, then read the guide below on how to build an agentic system.

Anthropic engineer: "Harness engineering is a really rare skill right now - and it's hard to know if you're even good at it." how he actually thinks about building the full system: • step 1 → the model. already smarter than we use it. "freeze development - you'd find new capabilities for another 6-12 months" • step 2 → the harness. delete the steering. "we stopped using plan mode - the model thinks correctly" • step 3 → the agents. workflows let Claude build its own harness per task and spin sub-agents. what didn't fit in one context window now runs across many • step 4 → the context. "RAG is an anti-pattern. use grep." memory via files, clean context window, prompt caching. that's the stack most people optimize step 1. he optimizes all four. Watch the full video, then read the guide below on how to build an agentic system.

Codez

44,590 views • 2 months ago

OpenAI’s hottest app isn’t ChatGPT—it’s Codex. In the last few weeks alone, the Codex team shipped a desktop app, GPT-5.3 Codex (a new flagship model), and Spark, the fastest coding model I’ve ever used. Usage has grown fivefold since January and over a million people now use Codex weekly. Codex was also the app that OpenAI chose to run an ad for in the Super Bowl. I talked to Thibault (Tibo), head of Codex, and Andrew (Andrew Ambrosino), a member of technical staff who built the Codex app, for Every 📧’s AI & I about what OpenAI is building and how they’re using it internally. We get into: - Why they built a GUI instead of a terminal. Terminals work for quick tasks, they say, but feel limiting when you’re running multiple agents in parallel. The IDE, meanwhile, overwhelms users—and the Codex team wants the AI to dynamically decide which tools to show you for a given task. - How they’re teaching the model to read between the lines. Codex is great at following instructions, but optimize too hard in that direction, and it starts taking you literally—like copying a typo directly into the code. The team obsesses over this tradeoff, and is also introducing “personalities,” modes users can toggle between that control how blunt or supportive the model feels. - How OpenAI uses its own coding agent. Codex lets you schedule prompts to run on a recurring basis, and the team has dozens of automations running at all times. For example, one scans for merge conflicts every couple of hours so code is always ready to ship, and another picks a random file from the codebase multiple times a day and hunts for bugs no one would've gone looking for. - Why speed is a dimension of intelligence. OpenAI’s newest model (Spark) is so fast that they actually slow it down so you can read the output. They see the speed enabling three things: staying super in the flow, replacing brittle developer tools with intelligent ones that can adapt on the fly, and redirecting the model mid-task— especially with voice—so coding starts to feel more and more like a conversation. - Code review is the next bottleneck. Models can generate code faster than ever, but someone still has to verify that it works. The team is exploring a future where the model proves its own fix works—retracing the click path a user would take, screenshotting the results, and attaching the evidence to a pull request. This is a must-watch for anyone who uses AI coding agents—and is curious about the future of programming. Watch below! Timestamps: Introduction: 00:01:27 OpenAI’s evolving bet on its coding agent: 00:05:27 The choice to invest in a GUI (over a terminal): 00:09:42 The AI workflows that the Codex team relies on to ship: 00:20:38 Teaching Codex how to read between the lines: 00:26:45 Building affordances for a lightening fast model: 00:28:45 Why speed is a dimension of intelligence: 00:33:15 Code review is the next bottleneck for coding agents: 00:36:30 How the Codex team positions against the competition: 00:41:24

OpenAI’s hottest app isn’t ChatGPT—it’s Codex. In the last few weeks alone, the Codex team shipped a desktop app, GPT-5.3 Codex (a new flagship model), and Spark, the fastest coding model I’ve ever used. Usage has grown fivefold since January and over a million people now use Codex weekly. Codex was also the app that OpenAI chose to run an ad for in the Super Bowl. I talked to Thibault (Tibo), head of Codex, and Andrew (Andrew Ambrosino), a member of technical staff who built the Codex app, for Every 📧’s AI & I about what OpenAI is building and how they’re using it internally. We get into: - Why they built a GUI instead of a terminal. Terminals work for quick tasks, they say, but feel limiting when you’re running multiple agents in parallel. The IDE, meanwhile, overwhelms users—and the Codex team wants the AI to dynamically decide which tools to show you for a given task. - How they’re teaching the model to read between the lines. Codex is great at following instructions, but optimize too hard in that direction, and it starts taking you literally—like copying a typo directly into the code. The team obsesses over this tradeoff, and is also introducing “personalities,” modes users can toggle between that control how blunt or supportive the model feels. - How OpenAI uses its own coding agent. Codex lets you schedule prompts to run on a recurring basis, and the team has dozens of automations running at all times. For example, one scans for merge conflicts every couple of hours so code is always ready to ship, and another picks a random file from the codebase multiple times a day and hunts for bugs no one would've gone looking for. - Why speed is a dimension of intelligence. OpenAI’s newest model (Spark) is so fast that they actually slow it down so you can read the output. They see the speed enabling three things: staying super in the flow, replacing brittle developer tools with intelligent ones that can adapt on the fly, and redirecting the model mid-task— especially with voice—so coding starts to feel more and more like a conversation. - Code review is the next bottleneck. Models can generate code faster than ever, but someone still has to verify that it works. The team is exploring a future where the model proves its own fix works—retracing the click path a user would take, screenshotting the results, and attaching the evidence to a pull request. This is a must-watch for anyone who uses AI coding agents—and is curious about the future of programming. Watch below! Timestamps: Introduction: 00:01:27 OpenAI’s evolving bet on its coding agent: 00:05:27 The choice to invest in a GUI (over a terminal): 00:09:42 The AI workflows that the Codex team relies on to ship: 00:20:38 Teaching Codex how to read between the lines: 00:26:45 Building affordances for a lightening fast model: 00:28:45 Why speed is a dimension of intelligence: 00:33:15 Code review is the next bottleneck for coding agents: 00:36:30 How the Codex team positions against the competition: 00:41:24

Dan Shipper 📧

15,588 views • 7 months ago

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and m_ric, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here!

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and m_ric, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here!

Andrew Ng

127,724 views • 1 year ago

Claude Code cracked something open for us Every 🧱. Now I ship to codebases I barely know, every feature we ship makes the next one easier, and non-technical members of the team use the terminal. I’m genuinely grateful. So I brought its creators, Cat Wu (cat) and Boris Cherny (Boris Cherny) from Anthropic, on AI & I to say thank you—and to talk about everything they’ve learned from building Claude Code. We get into: • The workflows Anthropic’s smartest engineers use to push Claude Code to its limits. Why they pit subagents against each other to get cleaner results, how they turn past code into leverage, and the slash commands and MCPs they rely on most. • The product lessons behind one of the most loved AI agents in the world. How the team balances simplicity and power—building a tool that anyone can use, but that experts can bend to their will—and their philosophy of “unshipping,” or cutting back whenever there’s a simpler, more intuitive path to user intent. • A peek into the future of coding with AI. The new form factors they’re experimenting with to make Claude Code more autonomous, more reliable, and more accessible to non-technical users This is a must-watch for anyone—both technical and non-technical—who wants to learn how to use Claude Code like the people who built it. Watch below! Timestamps: Introduction: 00:01:26 Claude Code’s origin story: 00:02:25 How Anthropic dogfoods Claude Code: 00:07:03 Boris and Cat’s favorite slash commands: 00:14:06 How Boris uses Claude Code to plan feature development: 00:15:49 Everything Anthropic has learned about using sub-agents well: 00:21:53 Use Claude Code to turn past code into leverage: 00:26:16 The product decisions for building an agent that’s simple and powerful: 00:33:14 Making Claude Code accessible to the non-technical user: 00:36:38 The next form factor for coding with AI: 00:45:12

Claude Code cracked something open for us Every 🧱. Now I ship to codebases I barely know, every feature we ship makes the next one easier, and non-technical members of the team use the terminal. I’m genuinely grateful. So I brought its creators, Cat Wu (cat) and Boris Cherny (Boris Cherny) from Anthropic, on AI & I to say thank you—and to talk about everything they’ve learned from building Claude Code. We get into: • The workflows Anthropic’s smartest engineers use to push Claude Code to its limits. Why they pit subagents against each other to get cleaner results, how they turn past code into leverage, and the slash commands and MCPs they rely on most. • The product lessons behind one of the most loved AI agents in the world. How the team balances simplicity and power—building a tool that anyone can use, but that experts can bend to their will—and their philosophy of “unshipping,” or cutting back whenever there’s a simpler, more intuitive path to user intent. • A peek into the future of coding with AI. The new form factors they’re experimenting with to make Claude Code more autonomous, more reliable, and more accessible to non-technical users This is a must-watch for anyone—both technical and non-technical—who wants to learn how to use Claude Code like the people who built it. Watch below! Timestamps: Introduction: 00:01:26 Claude Code’s origin story: 00:02:25 How Anthropic dogfoods Claude Code: 00:07:03 Boris and Cat’s favorite slash commands: 00:14:06 How Boris uses Claude Code to plan feature development: 00:15:49 Everything Anthropic has learned about using sub-agents well: 00:21:53 Use Claude Code to turn past code into leverage: 00:26:16 The product decisions for building an agent that’s simple and powerful: 00:33:14 Making Claude Code accessible to the non-technical user: 00:36:38 The next form factor for coding with AI: 00:45:12

Dan Shipper 📧

57,619 views • 11 months ago

Claude Code is the OpenClaw alternative you already had. I've been telling people this in person for three months. Now it's a piece on Every 📧. When OpenClaw was at peak hype a few months ago, everyone was going crazy over it — for good reasons. But almost nobody made the obvious comparison: Claude Code already does the thing you love about OpenClaw. In a lot of cases, better. Here's why people missed it. You have to understand the difference between a model and a harness. OpenClaw, at the end of the day, is just a harness on top of AI models. And Claude Code is too — it just got marketed as a coding tool, an alternative to Cursor. Until Cowork launched, almost nobody looked at it as a tool for non-programmers. Dan Shipper 📧 was one of the earliest to make that claim, and he was damn right. Let Claude Code run from the home folder of your computer with free access, and you get exactly what made OpenClaw exciting. Instead of just arguing it, here's a video walkthrough of a couple of my AI employees i built on the Claude Code harness. And here's the full article:

Claude Code is the OpenClaw alternative you already had. I've been telling people this in person for three months. Now it's a piece on Every 📧. When OpenClaw was at peak hype a few months ago, everyone was going crazy over it — for good reasons. But almost nobody made the obvious comparison: Claude Code already does the thing you love about OpenClaw. In a lot of cases, better. Here's why people missed it. You have to understand the difference between a model and a harness. OpenClaw, at the end of the day, is just a harness on top of AI models. And Claude Code is too — it just got marketed as a coding tool, an alternative to Cursor. Until Cowork launched, almost nobody looked at it as a tool for non-programmers. Dan Shipper 📧 was one of the earliest to make that claim, and he was damn right. Let Claude Code run from the home folder of your computer with free access, and you get exactly what made OpenClaw exciting. Instead of just arguing it, here's a video walkthrough of a couple of my AI employees i built on the Claude Code harness. And here's the full article:

Nityesh

22,895 views • 3 months ago

The entire AI industry is racing to build the smartest model. Satya Nadella just admitted that is not where the money is. The model is not the product. The harness is. That is the exact line. And it changes what Microsoft is actually competing on. OpenAI, Anthropic, Google, xAI, Meta every frontier lab is pouring hundreds of billions into training compute, chasing the next capability jump. Each betting that raw model intelligence is the moat. Microsoft is doing the opposite. It is building the harness the orchestration layer that sits above the model, connecting it to tools, data, permissions, sub-agents, and enterprise workflows. And it is letting OpenAI, Anthropic, and MAI compete to plug into it. "You need the model. But the model is not the product. The harness is." So do the math on what a harness actually does. A raw model dropped into an enterprise answers questions. That is a chatbot. A harness turns that same model into an agent that reads the SharePoint, edits the ERP entry, pulls the GitHub PR, updates Salesforce, and files the Excel report with the right permissions, the right audit trail, and the right sub-agent for each sub-task. The model provides the intelligence. The harness converts intelligence into work. Now here's where it gets interesting. "Even the best model in the world will feel broken without a great harness. And an okay model with a great harness can feel like magic." If that is true, the enterprise buyer is not buying model quality. The enterprise buyer is buying the harness. Which means model quality becomes a commodity input over time, and harness quality becomes the sustainable moat. Compare that to the strategy the entire frontier lab industry is executing. Everyone else is chasing the numerator raw intelligence. Almost nobody at scale is racing to build the denominator the orchestration layer that determines whether that intelligence can actually be deployed profitably inside a real company. The frontier model race has a 10 to 20 percent chance of producing a single dominant winner. Nadella just told the industry he does not need to be that winner. If OpenAI wins, Microsoft wins. If Anthropic wins, Microsoft wins. If MAI wins, Microsoft wins. If someone Microsoft has never heard of trains a better model in 2027, Microsoft still wins. Because the compute they train on, the harness they get plugged into, the enterprise contracts they get delivered through, and the products they sit inside are all Microsoft. He is not building the best AI model. He is building the layer that the best AI model has to run on to make anyone money. I wonder which position looks more valuable in ten years.

The entire AI industry is racing to build the smartest model. Satya Nadella just admitted that is not where the money is. The model is not the product. The harness is. That is the exact line. And it changes what Microsoft is actually competing on. OpenAI, Anthropic, Google, xAI, Meta every frontier lab is pouring hundreds of billions into training compute, chasing the next capability jump. Each betting that raw model intelligence is the moat. Microsoft is doing the opposite. It is building the harness the orchestration layer that sits above the model, connecting it to tools, data, permissions, sub-agents, and enterprise workflows. And it is letting OpenAI, Anthropic, and MAI compete to plug into it. "You need the model. But the model is not the product. The harness is." So do the math on what a harness actually does. A raw model dropped into an enterprise answers questions. That is a chatbot. A harness turns that same model into an agent that reads the SharePoint, edits the ERP entry, pulls the GitHub PR, updates Salesforce, and files the Excel report with the right permissions, the right audit trail, and the right sub-agent for each sub-task. The model provides the intelligence. The harness converts intelligence into work. Now here's where it gets interesting. "Even the best model in the world will feel broken without a great harness. And an okay model with a great harness can feel like magic." If that is true, the enterprise buyer is not buying model quality. The enterprise buyer is buying the harness. Which means model quality becomes a commodity input over time, and harness quality becomes the sustainable moat. Compare that to the strategy the entire frontier lab industry is executing. Everyone else is chasing the numerator raw intelligence. Almost nobody at scale is racing to build the denominator the orchestration layer that determines whether that intelligence can actually be deployed profitably inside a real company. The frontier model race has a 10 to 20 percent chance of producing a single dominant winner. Nadella just told the industry he does not need to be that winner. If OpenAI wins, Microsoft wins. If Anthropic wins, Microsoft wins. If MAI wins, Microsoft wins. If someone Microsoft has never heard of trains a better model in 2027, Microsoft still wins. Because the compute they train on, the harness they get plugged into, the enterprise contracts they get delivered through, and the products they sit inside are all Microsoft. He is not building the best AI model. He is building the layer that the best AI model has to run on to make anyone money. I wonder which position looks more valuable in ten years.

Vikram M

21,463 views • 3 months ago

Game development remains one of the most-requested, and most-challenging, categories on Arena. Wayne Chi, PhD candidate at Carnegie Mellon University and research intern at Arena, just walked us through GameDevBench: a benchmark built from real tutorials that turns game development into verifiable, deterministic tasks. How do top frontier models perform on tasks a human beginner could complete in under an hour, and is the biggest bottleneck coding or multimodal understanding? Check out Wayne’s full talk to hear results and learn more about GameDevBench:

Game development remains one of the most-requested, and most-challenging, categories on Arena. Wayne Chi, PhD candidate at Carnegie Mellon University and research intern at Arena, just walked us through GameDevBench: a benchmark built from real tutorials that turns game development into verifiable, deterministic tasks. How do top frontier models perform on tasks a human beginner could complete in under an hour, and is the biggest bottleneck coding or multimodal understanding? Check out Wayne’s full talk to hear results and learn more about GameDevBench:

Arena.ai

17,370 views • 1 month ago

GREG ISENBERG JUST INTERVIEWED A SOLO FOUNDER WHO MAKES $1,000,000 A YEAR BUILDING AI AGENTS FOR REAL ESTATE AGENTS the agents cost $5,000 to $10,000 a month per client. boring industries pay the most. real estate, insurance, manufacturing, legal. they have money and they hate using ai he uses obsidian as his second brain because notion can't run agents that actually do work for him his entire stack: granola, trello, loom, superhuman, asana, codex, hermes, orgo, composio, agent mail. gpt 5.5 by default. claude opus for long coding sessions 99% of the market doesn't know what an ai agent is yet. that's the moat. you don't need to be the smartest. you need to be the one who calls them first the full breakdown is in the 45 minute video and it's worth every minute bookmark this and read the article below

GREG ISENBERG JUST INTERVIEWED A SOLO FOUNDER WHO MAKES $1,000,000 A YEAR BUILDING AI AGENTS FOR REAL ESTATE AGENTS the agents cost $5,000 to $10,000 a month per client. boring industries pay the most. real estate, insurance, manufacturing, legal. they have money and they hate using ai he uses obsidian as his second brain because notion can't run agents that actually do work for him his entire stack: granola, trello, loom, superhuman, asana, codex, hermes, orgo, composio, agent mail. gpt 5.5 by default. claude opus for long coding sessions 99% of the market doesn't know what an ai agent is yet. that's the moat. you don't need to be the smartest. you need to be the one who calls them first the full breakdown is in the 45 minute video and it's worth every minute bookmark this and read the article below

starmex

497,026 views • 4 months ago

New course to bring you up to state-of-the-art at using AI to help you code: Build Apps with Windsurf's AI Coding Agents, built in partnership with WIndsurf (Codeium) and taught by Anshul Ramachandran! AI-assisted IDEs (Integrated Development Environments) make developers’ workflows faster, more efficient, and much more fun. Agentic tools like Windsurf are more than just code autocomplete—they are collaborative coding agents that help you break down complex applications, iterate efficiently, and generate code that spans multiple files. Although a lot of coding assistants share the same underlying large language models for planning and reasoning, a major point of distinction is how they handle tools, keep track of context, and stay aligned with your intent as a developer. For instance, if you make modifications to a class definition in your code and make the same modifications to other classes in the same directory, you might tell the AI agent "Do the same thing in similar places in this directory." Here, tracking your intent means understanding that “the same thing" refers to that recent edit you just made, which must be followed by appropriate search and tool-calling to implement the changes. In this course, you'll learn the inner workings of coding agents, their strengths and limitations, and how to use Windsurf to quickly build several applications. In detail, you'll: - Build a mental model of how agents work by combining human-action tracking, tool integration, and context awareness to carry out an agentic coding workflow. - Learn the challenges of code search and discovery and how a multi-step retrieval approach helps coding agents address them. - Use Windsurf to analyze and understand a large, old codebase and update it to the latest versions of the frameworks and packages it uses. - Build a Wikipedia data analysis app that retrieves, parses, and analyzes word frequencies. - Enhance the performance of your Wikipedia analysis app by adding caching, and through this, also learn how to course-correct when the AI agent produces unexpected results. - Learn tips and tricks such as keyboard shortcuts, autocomplete, and @ mentions to quickly call on agentic capabilities. - Use image/multimodal capabilities of the AI agent to increase your development velocity; you'll see an example of uploading a mockup with sketched-out UI features, and ask the agent to use that to build new functionality to an app. By the end of this course, you’ll understand agentic coding in-depth and know how to use it to make your development process much faster, more efficient, and enjoyable. Please sign up here!

New course to bring you up to state-of-the-art at using AI to help you code: Build Apps with Windsurf's AI Coding Agents, built in partnership with WIndsurf (Codeium) and taught by Anshul Ramachandran! AI-assisted IDEs (Integrated Development Environments) make developers’ workflows faster, more efficient, and much more fun. Agentic tools like Windsurf are more than just code autocomplete—they are collaborative coding agents that help you break down complex applications, iterate efficiently, and generate code that spans multiple files. Although a lot of coding assistants share the same underlying large language models for planning and reasoning, a major point of distinction is how they handle tools, keep track of context, and stay aligned with your intent as a developer. For instance, if you make modifications to a class definition in your code and make the same modifications to other classes in the same directory, you might tell the AI agent "Do the same thing in similar places in this directory." Here, tracking your intent means understanding that “the same thing" refers to that recent edit you just made, which must be followed by appropriate search and tool-calling to implement the changes. In this course, you'll learn the inner workings of coding agents, their strengths and limitations, and how to use Windsurf to quickly build several applications. In detail, you'll: - Build a mental model of how agents work by combining human-action tracking, tool integration, and context awareness to carry out an agentic coding workflow. - Learn the challenges of code search and discovery and how a multi-step retrieval approach helps coding agents address them. - Use Windsurf to analyze and understand a large, old codebase and update it to the latest versions of the frameworks and packages it uses. - Build a Wikipedia data analysis app that retrieves, parses, and analyzes word frequencies. - Enhance the performance of your Wikipedia analysis app by adding caching, and through this, also learn how to course-correct when the AI agent produces unexpected results. - Learn tips and tricks such as keyboard shortcuts, autocomplete, and @ mentions to quickly call on agentic capabilities. - Use image/multimodal capabilities of the AI agent to increase your development velocity; you'll see an example of uploading a mockup with sketched-out UI features, and ask the agent to use that to build new functionality to an app. By the end of this course, you’ll understand agentic coding in-depth and know how to use it to make your development process much faster, more efficient, and enjoyable. Please sign up here!

Andrew Ng

140,233 views • 1 year ago