Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Overview of our recent launch of Coding Agent benchmarks on Artificial Analysis and our first Youtube Video! We walk through the performance, cost, token usage and speed differences across different coding agents. This includes looking at Opus 4.7 in Claude Code's leading performance and Composer 2.5's strong positioning on... show more

Artificial Analysis

130,534 subscribers

10,702 views • 5 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Our new Gemini 3.5 Flash is our strongest agentic and coding model yet, delivering frontier-level performance at 4x the speed of comparable frontier models at less than half the cost. We had Tulsee Doshi break down what you need to know about it.

Our new Gemini 3.5 Flash is our strongest agentic and coding model yet, delivering frontier-level performance at 4x the speed of comparable frontier models at less than half the cost. We had Tulsee Doshi break down what you need to know about it.

Google

77,882 views • 5 months ago

Announcing Grok 4.5, our first model trained specifically for coding and agents. It was trained with Cursor and offers frontier intelligence at leading speeds and cost efficiency.

Announcing Grok 4.5, our first model trained specifically for coding and agents. It was trained with Cursor and offers frontier intelligence at leading speeds and cost efficiency.

SpaceX Super Intelligence

69,307,776 views • 4 months ago

Introducing Duet CLI. Now coding agents like Claude Code and Codex can call Duet (our agent-building agent) directly from the terminal to build and modify Decagon agents. Duet handles work within Decagon and returns the result while your coding agent stays focused on the broader task.

Introducing Duet CLI. Now coding agents like Claude Code and Codex can call Duet (our agent-building agent) directly from the terminal to build and modify Decagon agents. Duet handles work within Decagon and returns the result while your coding agent stays focused on the broader task.

Decagon

770,006 views • 3 months ago

Agent Categories and Costs are now in the Agent Leaderboard! Two questions guide model selection: task-specific performance and cost to completion. Based on analysis of 1.7M+ real-world Agent Mode sessions, the leaderboard now lets users: - Filter the leaderboard by Code, Work, and Chat to compare model performance by task type - View real cost-per-task for each model, including the Pareto frontier of the most cost-efficient options at each performance level

Agent Categories and Costs are now in the Agent Leaderboard! Two questions guide model selection: task-specific performance and cost to completion. Based on analysis of 1.7M+ real-world Agent Mode sessions, the leaderboard now lets users: - Filter the leaderboard by Code, Work, and Chat to compare model performance by task type - View real cost-per-task for each model, including the Pareto frontier of the most cost-efficient options at each performance level

Arena.ai

29,605 views • 2 months ago

🚨 Create Your Own Custom Coding Agent - Mix Fable 5, GPT 5.6 sol and Grok 4.5! Our newest feature allows you to mix and match your favorite LLMs and create your custom coding agent Create your favorite cocktail such as - Fable 5 for hard coding - 5.6 sol for backend - Opus 4.8 for frontend - Grok 4.5 for easy coding You can also only choose cheap LLMs for a cost optimized experience. Your use custom agent in API, chat or our agent platform

🚨 Create Your Own Custom Coding Agent - Mix Fable 5, GPT 5.6 sol and Grok 4.5! Our newest feature allows you to mix and match your favorite LLMs and create your custom coding agent Create your favorite cocktail such as - Fable 5 for hard coding - 5.6 sol for backend - Opus 4.8 for frontend - Grok 4.5 for easy coding You can also only choose cheap LLMs for a cost optimized experience. Your use custom agent in API, chat or our agent platform

Bindu Reddy

7,810,140 views • 4 months ago

Meet Agent Bake-Off: blind side by side tests for coding agents like Claude Code, Cursor , Amp , Factory , Codex, & more. We've been building the largest real world benchmarks for coding agents, and we're excited share an early preview of our open community tooling.

Meet Agent Bake-Off: blind side by side tests for coding agents like Claude Code, Cursor , Amp , Factory , Codex, & more. We've been building the largest real world benchmarks for coding agents, and we're excited share an early preview of our open community tooling.

brexton

18,309 views • 1 year ago

Announcing Opcode Everyone's favorite Claude Code GUI Claudia is now Opcode... with a complete revamp. And we just hit 15K+ stars on GitHub! The support from the community has been immense! ❤️ What's new: - Feature parity with all new Claude Code features in the CLI. - A complete UI revamp, faster and cleaner. What's coming: - Opcode's own coding agent that exclusively uses open models while achieving SOTA performance on coding benchmarks. We're currently piloting this at companies requiring fully local on-prem coding agents. If you're one of them, reach out to us and we'll set this up for you! Try Opcode at opcode [dot] sh Like/retweet and reply to this tweet with "join" to get an invite to the Discord server and beta test our coding agent for free.

Announcing Opcode Everyone's favorite Claude Code GUI Claudia is now Opcode... with a complete revamp. And we just hit 15K+ stars on GitHub! The support from the community has been immense! ❤️ What's new: - Feature parity with all new Claude Code features in the CLI. - A complete UI revamp, faster and cleaner. What's coming: - Opcode's own coding agent that exclusively uses open models while achieving SOTA performance on coding benchmarks. We're currently piloting this at companies requiring fully local on-prem coding agents. If you're one of them, reach out to us and we'll set this up for you! Try Opcode at opcode [dot] sh Like/retweet and reply to this tweet with "join" to get an invite to the Discord server and beta test our coding agent for free.

winfunc

38,930 views • 1 year ago

Introducing GLM-5V-Turbo: Vision Coding Model - Native Multimodal Coding: Natively understands multimodal inputs including images, videos, design drafts, and document layouts. - Balanced Visual and Programming Capabilities: Achieves leading performance across core benchmarks for multimodal coding, tool use, and GUI Agents. - Deep Adaptation for Claude Code and Claw Scenarios: Works in deep synergy with Agents like Claude Code and OpenClaw. Try it now: API: Coding Plan trial applications:

Introducing GLM-5V-Turbo: Vision Coding Model - Native Multimodal Coding: Natively understands multimodal inputs including images, videos, design drafts, and document layouts. - Balanced Visual and Programming Capabilities: Achieves leading performance across core benchmarks for multimodal coding, tool use, and GUI Agents. - Deep Adaptation for Claude Code and Claw Scenarios: Works in deep synergy with Agents like Claude Code and OpenClaw. Try it now: API: Coding Plan trial applications:

Z.ai

1,971,324 views • 7 months ago

GLM-4.7 from Z.ai is live on Cerebras! - Frontier intelligence for coding, tool-driven agents, and multi-turn reasoning - Record coding speed: ~1,000 tokens per second (up to 1,700 TPS for other uses) - Strong price-performance: ~10x higher than Sonnet 4.5

GLM-4.7 from Z.ai is live on Cerebras! - Frontier intelligence for coding, tool-driven agents, and multi-turn reasoning - Record coding speed: ~1,000 tokens per second (up to 1,700 TPS for other uses) - Strong price-performance: ~10x higher than Sonnet 4.5

Cerebras

135,149 views • 10 months ago

ICYMI, GitHub Copilot coding agent now works with Claude Sonnet 4.5 - which you can leverage through Visual Studio Code's deep coding agent integration.

ICYMI, GitHub Copilot coding agent now works with Claude Sonnet 4.5 - which you can leverage through Visual Studio Code's deep coding agent integration.

Visual Studio Code

33,996 views • 1 year ago

$Everyone's sleeping on MiniMax. Again. They just shipped M3. The first open-weights model to combine frontier coding, 1M context, and native multimodality in one drop. I plugged it into Claude Code this morning. Pasted a design from Dribbble. Watched M3 write production-ready React code in one session. At the agency, I just replaced Opus 4.8 with M3 for 80% of our coding tasks. The output is the same and we are running everything at a fraction of the cost. Open infrastructure is the future.$

Everyone's sleeping on MiniMax. Again. They just shipped M3. The first open-weights model to combine frontier coding, 1M context, and native multimodality in one drop. I plugged it into Claude Code this morning. Pasted a design from Dribbble. Watched M3 write production-ready React code in one session. At the agency, I just replaced Opus 4.8 with M3 for 80% of our coding tasks. The output is the same and we are running everything at a fraction of the cost. Open infrastructure is the future.

Prajwal Tomar

13,063 views • 5 months ago

A conversation with Claude Code's creator Boris Cherny on the future of agentic coding, the evolution of coding models, and how he uses Claude Code to build Claude Code.

A conversation with Claude Code's creator Boris Cherny on the future of agentic coding, the evolution of coding models, and how he uses Claude Code to build Claude Code.

Alex Albert

81,455 views • 1 year ago

Claude Code vs Codex vs Pi: which coding agent wins? Melissa Pan, PhD candidate at UC Berkeley’s Sky Computing Lab and previous Arena.ai intern, explored the hidden “harness tax”: how the system surrounding an AI model affects its cost and performance. She reports three surprising findings. One: harness choice impacts cost more than accuracy. To learn about all three findings, and what they mean for building useful coding agents on realistic budgets, check out the full video here:

Claude Code vs Codex vs Pi: which coding agent wins? Melissa Pan, PhD candidate at UC Berkeley’s Sky Computing Lab and previous Arena.ai intern, explored the hidden “harness tax”: how the system surrounding an AI model affects its cost and performance. She reports three surprising findings. One: harness choice impacts cost more than accuracy. To learn about all three findings, and what they mean for building useful coding agents on realistic budgets, check out the full video here:

Arena.ai

85,347 views • 1 month ago

Introducing Shift-1, a frontier coding router trained on AC2 that reduced coding agent cost by up to 55% with no noticeable drop in quality. Try it at

Introducing Shift-1, a frontier coding router trained on AC2 that reduced coding agent cost by up to 55% with no noticeable drop in quality. Try it at

Applied Compute

156,699 views • 22 days ago

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

Artificial Analysis

13,214 views • 4 months ago

Introducing Reason 1.0 A frontier coding agent for valuable work. We wanted to see what a good coding agent would look like. Seamless, simple with customization for your models and subscriptions. Fast and lightweight, Reason Scores incredible on standard benchmarks. Use it for free today

Introducing Reason 1.0 A frontier coding agent for valuable work. We wanted to see what a good coding agent would look like. Seamless, simple with customization for your models and subscriptions. Fast and lightweight, Reason Scores incredible on standard benchmarks. Use it for free today

Adi Singh

51,255 views • 1 month ago

🔧 Let your coding agent identify and fix INP issues for you → With Chrome DevTools MCP, your coding agent can measure performance for specific user interactions to find and resolve slow UI responses - on its own with the help of the browser!

🔧 Let your coding agent identify and fix INP issues for you → With Chrome DevTools MCP, your coding agent can measure performance for specific user interactions to find and resolve slow UI responses - on its own with the help of the browser!

Chrome for Developers

16,721 views • 1 year ago

Embedder is the world's first hardware-aware coding agent. By understanding and interacting directly with your hardware, it achieves state of the art performance in an embedded systems (C++/Rust) context. Our latest update (v0.3.0) features a stunning new terminal UI, and our fastest, most capable firmware agent yet: Embedder

Embedder is the world's first hardware-aware coding agent. By understanding and interacting directly with your hardware, it achieves state of the art performance in an embedded systems (C++/Rust) context. Our latest update (v0.3.0) features a stunning new terminal UI, and our fastest, most capable firmware agent yet: Embedder

Ethan Gibbs

435,583 views • 9 months ago

Gemini 3.5 Flash is a step forward for Google on speed and agentic capabilities but comes at a trade-off of being higher cost than prior models We have measured up to ~280 output tokens/sec, placing it on the speed/intelligence Pareto frontier and well ahead of Gemini 3 Flash. It also shows a major uplift on agentic tasks, reaching ~1650 ELO on GDPVal-AA. The trade-off: cost is up ~5x versus Gemini 3 Flash, driven by higher token prices (3x higher than Gemini 3 Flash) and higher token usage. In this video, Declan Jackson, Member of Technical Staff at Artificial Analysis, breaks it down.

Gemini 3.5 Flash is a step forward for Google on speed and agentic capabilities but comes at a trade-off of being higher cost than prior models We have measured up to ~280 output tokens/sec, placing it on the speed/intelligence Pareto frontier and well ahead of Gemini 3 Flash. It also shows a major uplift on agentic tasks, reaching ~1650 ELO on GDPVal-AA. The trade-off: cost is up ~5x versus Gemini 3 Flash, driven by higher token prices (3x higher than Gemini 3 Flash) and higher token usage. In this video, Declan Jackson, Member of Technical Staff at Artificial Analysis, breaks it down.

Artificial Analysis

16,055 views • 5 months ago