Artificial Analysis's banner

Artificial Analysis

@ArtificialAnlys • 178,284 subscribers

Independent analysis of AI

Shorts

Gemini 4 Carbon broke one of our arms…. Gemini 4 Carbon spent 7 hours and 33 minutes in the AA-Robotics v0.1 lab, bashing the arm around so much it broke a gripper (we gave it a retry on that one)

Gemini 4 Carbon broke one of our arms…. Gemini 4 Carbon spent 7 hours and 33 minutes in the AA-Robotics v0.1 lab, bashing the arm around so much it broke a gripper (we gave it a retry on that one)

40,180 views

Claude Fable 5.2 is now #1 on AA-3D, our benchmark for how well AI models build interactive 3D experiences Claude Fable 5.2 (Max) scores 1271 Elo, 67 points ahead of Claude Sonnet 5.5 (1204) and Claude Opus 5.5 (1202). Its 95% confidence interval (1246 to 1297) does not overlap any other models, so its lead is outside the margin of error. Key results: ➤ It scores 132 points above its predecessor, Claude Fable 5.1 (1139) ➤ It is ahead of the highest-ranked OpenAI and Google models on AA-3D, GPT-6 Astra (1168) and Gemini 4 Argon (1147) More results and scenes in the thread below

Claude Fable 5.2 is now #1 on AA-3D, our benchmark for how well AI models build interactive 3D experiences Claude Fable 5.2 (Max) scores 1271 Elo, 67 points ahead of Claude Sonnet 5.5 (1204) and Claude Opus 5.5 (1202). Its 95% confidence interval (1246 to 1297) does not overlap any other models, so its lead is outside the margin of error. Key results: ➤ It scores 132 points above its predecessor, Claude Fable 5.1 (1139) ➤ It is ahead of the highest-ranked OpenAI and Google models on AA-3D, GPT-6 Astra (1168) and Gemini 4 Argon (1147) More results and scenes in the thread below

33,863 views

Claude Fable 5.2 leads scientific visual reasoning benchmark VIALS-AA; it makes more tool calls than any other Anthropic model, unlike OpenAI models which make fewer as they get smarter VIALS-AA is our implementation of Handshake AI’s VIALS benchmark for visual reasoning in life sciences research, across artifacts such as gel blots, plasmid maps, and phylogenies. Claude Fable 5.2 (max with fallback) scores 67% on VIALS-AA, ahead of Gemini 4 Carbon (high) at 64% and GPT-6 Astra (max) at 63%. Fable averages ~37 tool calls per task, while Astra averages ~8, the fewest of any model we’ve tested. On turn 4 of this protein interaction task, Fable creates crops and asks to view them. On turn 10, it builds a color mask by thresholding color channels. On turn 17, it fits a circle to each arc and unrolls the arc into a straight strip. On turn 31, it counts the bars as peaks in an intensity profile. And on turn 41, it maps every detected spoke back on the original image to check its work. Fable batches calls on 28 of its 51 turns, averaging 1.75 tool calls per turn, compared with 1.12 for Sonnet 5.5.

22,143 views

We handed GPT-6.1 Sol a robot arm and a pen, and it wrote the three scariest words in AI Happy Halloween from AA-Robotics! 🎃

We handed GPT-6.1 Sol a robot arm and a pen, and it wrote the three scariest words in AI Happy Halloween from AA-Robotics! 🎃

65,834 views

Same robot, same task, two models with very different writing styles ➤ On the left, Claude Opus 5.5 dabs out "hello world" dot by dot: 3,270 arm moves over 34 minutes ➤ On the right, GPT-6 Astra writes in continuous strokes: 514 moves in 13 minutes From AA-Robotics v0.1, seen from the model’s view See more examples here:

Same robot, same task, two models with very different writing styles ➤ On the left, Claude Opus 5.5 dabs out "hello world" dot by dot: 3,270 arm moves over 34 minutes ➤ On the right, GPT-6 Astra writes in continuous strokes: 514 moves in 13 minutes From AA-Robotics v0.1, seen from the model’s view See more examples here:

62,902 views

Knowing when to stop is hard for language models controlling a robot arm ➤ Claude Opus 5.5 completed the task in 23 minutes, and said so: "The crocodile looks fully inside now" ➤ It then spent 22 minutes pushing the crocodile back and forth, only to finish almost exactly where it was before

Knowing when to stop is hard for language models controlling a robot arm ➤ Claude Opus 5.5 completed the task in 23 minutes, and said so: "The crocodile looks fully inside now" ➤ It then spent 22 minutes pushing the crocodile back and forth, only to finish almost exactly where it was before

36,938 views

Fish Audio Drama 3 is the #1 open weights model on both Artificial Analysis Speech Arenas, and #1 of all models on Pronunciation Robustness Fish Audio has released Drama 3. We evaluated it across both Speech Arenas and our Pronunciation Robustness benchmark. Key takeaways: ➤ Text to Speech, Provider Voice: Drama 3 scores an Elo of 1,278 across 1,512 arena appearances, ranking #6 overall and #1 among open weights models, 38 points clear of its own Drama 3 Preview at 1,240 and 50 ahead of BreezeBlue's Breeze TTS 2 at 1,228 ➤ Text to Speech, Controlled Voice: Drama 3 scores 1,085 across 1,958 appearances, ranking #12 overall and #1 among open weights models, ahead of Ant Group's Ming-3.0-tts at 1,068 and its own Preview at 1,063 ➤ Pronunciation Robustness: Drama 3 scores 94.6% across 2,157 rated spans, ranking #1, ahead of Ming-3.0-tts and Drama 3 Preview, both at 93.2% ➤ Speed: Drama 3 generates 66.6 characters per second, ahead of its own Drama 3 Preview at 60.7 and Murf's Speech Gen 2 at 63.0, and behind ElevenLabs' Eleven v4 at 70.0 and v3 Conversational at 66.8 ➤ Pricing: Drama 3 is priced at $15/1M characters. Every model ranked above it on Provider Voice costs more: Gemini 3.8 Flash TTS at $16.5/1M, Qwen-Audio-3.1-TTS-Plus at $19.3/1M, Eleven v4 Turbo at $40/1M, Sonic 3.6 at $49/1M and Eleven v4 at $80/1M ➤ Open weights: released under the Fish Audio Community License. See more details and listen to samples below ⬇️

Fish Audio Drama 3 is the #1 open weights model on both Artificial Analysis Speech Arenas, and #1 of all models on Pronunciation Robustness Fish Audio has released Drama 3. We evaluated it across both Speech Arenas and our Pronunciation Robustness benchmark. Key takeaways: ➤ Text to Speech, Provider Voice: Drama 3 scores an Elo of 1,278 across 1,512 arena appearances, ranking #6 overall and #1 among open weights models, 38 points clear of its own Drama 3 Preview at 1,240 and 50 ahead of BreezeBlue's Breeze TTS 2 at 1,228 ➤ Text to Speech, Controlled Voice: Drama 3 scores 1,085 across 1,958 appearances, ranking #12 overall and #1 among open weights models, ahead of Ant Group's Ming-3.0-tts at 1,068 and its own Preview at 1,063 ➤ Pronunciation Robustness: Drama 3 scores 94.6% across 2,157 rated spans, ranking #1, ahead of Ming-3.0-tts and Drama 3 Preview, both at 93.2% ➤ Speed: Drama 3 generates 66.6 characters per second, ahead of its own Drama 3 Preview at 60.7 and Murf's Speech Gen 2 at 63.0, and behind ElevenLabs' Eleven v4 at 70.0 and v3 Conversational at 66.8 ➤ Pricing: Drama 3 is priced at $15/1M characters. Every model ranked above it on Provider Voice costs more: Gemini 3.8 Flash TTS at $16.5/1M, Qwen-Audio-3.1-TTS-Plus at $19.3/1M, Eleven v4 Turbo at $40/1M, Sonic 3.6 at $49/1M and Eleven v4 at $80/1M ➤ Open weights: released under the Fish Audio Community License. See more details and listen to samples below ⬇️

32,328 views

Explore AA-3D scenes in your browser: pick any of the tasks and switch between the models to compare them AA-3D Explorer lets you choose a brief from one of the seven domains, then step through the models to see how each one built the same scene. Steer, fly, and click through each scene, as our human panel does before it votes.

Explore AA-3D scenes in your browser: pick any of the tasks and switch between the models to compare them AA-3D Explorer lets you choose a brief from one of the seven domains, then step through the models to see how each one built the same scene. Steer, fly, and click through each scene, as our human panel does before it votes.

26,111 views

We gave AI models one prompt: build an interactive black hole, physically informed, with the presence of Gargantua from Interstellar.

We gave AI models one prompt: build an interactive black hole, physically informed, with the presence of Gargantua from Interstellar.

40,948 views

You can test GPT-6 Astra on your own custom benchmarks in Optima Head to to start creating and running your own custom benchmarks today

You can test GPT-6 Astra on your own custom benchmarks in Optima Head to to start creating and running your own custom benchmarks today

18,129 views

You can now benchmark Qwen3.8 27B with Optima. See how a model that can run on your laptop compares on performance, cost and speed for your custom use cases We're also offering credits and free advisory from an Artificial Analysis team member to enterprises looking to build benchmarks for their use cases. Let us know if you’re interested: Build and run your custom benchmarks today at

You can now benchmark Qwen3.8 27B with Optima. See how a model that can run on your laptop compares on performance, cost and speed for your custom use cases We're also offering credits and free advisory from an Artificial Analysis team member to enterprises looking to build benchmarks for their use cases. Let us know if you’re interested: Build and run your custom benchmarks today at

15,947 views

Want to generate images from the world's best models like Nano Banana or GPT Image side by side? Now you can with Image Lab 🚀 Run a single prompt across up to 25 models, with up to 20 images from each, and see results in seconds. You've seen our leaderboards. Now generate and evaluate the models yourself. Link in thread 🧵👇

Want to generate images from the world's best models like Nano Banana or GPT Image side by side? Now you can with Image Lab 🚀 Run a single prompt across up to 25 models, with up to 20 images from each, and see results in seconds. You've seen our leaderboards. Now generate and evaluate the models yourself. Link in thread 🧵👇

12,250 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Gemini 4 Hydrogen is the leading Google model on AA-Robotics v0.1, sitting fourth overall behind Claude Fable 5.2 (75), GPT-6 Astra (69) and GPT-6.1 Sol (65) Hydrogen was very honest about its performance, pointing out a minor spill when pouring the drink and an unclosed lid when tidying blocks

Gemini 4 Hydrogen is the leading Google model on AA-Robotics v0.1, sitting fourth overall behind Claude Fable 5.2 (75), GPT-6 Astra (69) and GPT-6.1 Sol (65) Hydrogen was very honest about its performance, pointing out a minor spill when pouring the drink and an unclosed lid when tidying blocks

Artificial Analysis

20,136 views • 11 hours ago

Claude Fable 5.2 is the new leader on AA-Robotics v0.1, scoring 75, ahead of GPT-6 Astra (69) and GPT-6.1 Sol (65) Fable 5.2 is the first model to achieve a score on the measure box task. It did so using purely video, leaving the tape measure as setup and using its markings as a scale to estimate the box size

Claude Fable 5.2 is the new leader on AA-Robotics v0.1, scoring 75, ahead of GPT-6 Astra (69) and GPT-6.1 Sol (65) Fable 5.2 is the first model to achieve a score on the measure box task. It did so using purely video, leaving the tape measure as setup and using its markings as a scale to estimate the box size

Artificial Analysis

26,011 views • 1 day ago

We gave four frontier image editing models the same photo and 30 edits in a row: Ideogram 4.5 keeps most of the room intact, while the others drift, GPT Image 2.5 Sunburst most visibly. We've seen some interesting demos of multi-turn editing consistency from the latest image editing models, so we ran our own test: GPT Image 2.5 Sunburst, #1 on our Image Editing leaderboard, against Ideogram 4.5, FLUX 3 and Nano Banana 2.1. Our leaderboard scores single edits; this tests what happens when 30 consecutive changes stack up, with each model editing its own previous output through a 30-step real estate staging sequence: light the fire, add a sofa, repaint the walls, swap day for twilight, and more. Why are the final results so different? Ideogram's Ideogram 4.5 and BlackForestLabsAI - Unofficial's FLUX 3 edit locally. On small edits like adding a vase of tulips, we measured that they left 95% or more of the image essentially untouched. GPT Image 2.5 (Sunburst) re-renders most of the entire image on every edit, leaving only about a fifth of the image unchanged, so small shifts in colour and detail compound over turns. Nano Banana 2.1 sits in between: its edits stay local, but the rest of the image shifts slightly and gradually darkens.

We gave four frontier image editing models the same photo and 30 edits in a row: Ideogram 4.5 keeps most of the room intact, while the others drift, GPT Image 2.5 Sunburst most visibly. We've seen some interesting demos of multi-turn editing consistency from the latest image editing models, so we ran our own test: GPT Image 2.5 Sunburst, #1 on our Image Editing leaderboard, against Ideogram 4.5, FLUX 3 and Nano Banana 2.1. Our leaderboard scores single edits; this tests what happens when 30 consecutive changes stack up, with each model editing its own previous output through a 30-step real estate staging sequence: light the fire, add a sofa, repaint the walls, swap day for twilight, and more. Why are the final results so different? Ideogram's Ideogram 4.5 and BlackForestLabsAI - Unofficial's FLUX 3 edit locally. On small edits like adding a vase of tulips, we measured that they left 95% or more of the image essentially untouched. GPT Image 2.5 (Sunburst) re-renders most of the entire image on every edit, leaving only about a fifth of the image unchanged, so small shifts in colour and detail compound over turns. Nano Banana 2.1 sits in between: its edits stay local, but the rest of the image shifts slightly and gradually darkens.

Artificial Analysis

94,163 views • 1 month ago

Kimi K3.1 Pro is among the leading models in agentic knowledge work and coding, but early signs suggest it will rank lower on AA-3D We are still collecting the preference votes for its Elo on AA-3D, our benchmark for how well AI models build interactive 3D scenes. In agentic knowledge work and coding, it is among the leading models on GDPval-AA and Terminal-Bench 4.0. Its predecessor, Kimi K3, is 12th of 22 models on AA-3D. Here is one of its AA-3D scenes: a 492 m mountain-bike descent with berms, roots, a rock garden, a drop, and a jump. A demo run rides it from top to bottom, and you can take over the steering, pedaling, and braking at any point. Congratulations to Kimi.ai on the release!

Kimi K3.1 Pro is among the leading models in agentic knowledge work and coding, but early signs suggest it will rank lower on AA-3D We are still collecting the preference votes for its Elo on AA-3D, our benchmark for how well AI models build interactive 3D scenes. In agentic knowledge work and coding, it is among the leading models on GDPval-AA and Terminal-Bench 4.0. Its predecessor, Kimi K3, is 12th of 22 models on AA-3D. Here is one of its AA-3D scenes: a 492 m mountain-bike descent with berms, roots, a rock garden, a drop, and a jump. A demo run rides it from top to bottom, and you can take over the steering, pedaling, and braking at any point. Congratulations to Kimi.ai on the release!

Artificial Analysis

34,408 views • 20 days ago

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

133,952 views • 3 months ago

Welcome to the era of embodied AI. We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today. We find that frontier language models in coding agent harnesses have recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly. GPT-6 Astra places first in AA-Robotics v0.1, followed by GPT-6.1 Sol, Gemini 4 Argon and Claude Opus 5.5. AA-Robotics v0.1 details: ➤ Our first eight tasks test a range of capabilities: control and dexterity (precise grasping and object handling), spatial reasoning (how objects interact with one another) and adaptability (how models respond to the unexpected) ➤ Models get only brief instructions: how to connect to the robot arm (via a simple HTTP API), the arm's name and how to submit an attempt. They can view the scene through four cameras and write and run code to plan and send movements ➤ Once a model submits an attempt, its end state is graded against the task's pass criteria. A pass scores 100; an incomplete attempt earns partial credit. All attempts are zero-shot Key findings: ➤ The GPT family tops the leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. Claude Opus 5.5 (49) and Claude Fable 5.1 (46) trail ➤ Today’s models are incredibly slow at our tasks, taking ~30 minutes on average to complete each tabletop task ➤ Cost varies widely: GPT-6.1 Sol is the cheapest at $1.93 per task on average; GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much ➤ Compared with a human, today's best-performing frontier language models are ~100x slower and ~10x more expensive More results, clips, and how AA-Robotics v0.1 works follow in the thread below.

Welcome to the era of embodied AI. We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today. We find that frontier language models in coding agent harnesses have recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly. GPT-6 Astra places first in AA-Robotics v0.1, followed by GPT-6.1 Sol, Gemini 4 Argon and Claude Opus 5.5. AA-Robotics v0.1 details: ➤ Our first eight tasks test a range of capabilities: control and dexterity (precise grasping and object handling), spatial reasoning (how objects interact with one another) and adaptability (how models respond to the unexpected) ➤ Models get only brief instructions: how to connect to the robot arm (via a simple HTTP API), the arm's name and how to submit an attempt. They can view the scene through four cameras and write and run code to plan and send movements ➤ Once a model submits an attempt, its end state is graded against the task's pass criteria. A pass scores 100; an incomplete attempt earns partial credit. All attempts are zero-shot Key findings: ➤ The GPT family tops the leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. Claude Opus 5.5 (49) and Claude Fable 5.1 (46) trail ➤ Today’s models are incredibly slow at our tasks, taking ~30 minutes on average to complete each tabletop task ➤ Cost varies widely: GPT-6.1 Sol is the cheapest at $1.93 per task on average; GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much ➤ Compared with a human, today's best-performing frontier language models are ~100x slower and ~10x more expensive More results, clips, and how AA-Robotics v0.1 works follow in the thread below.

Artificial Analysis

38,735 views • 29 days ago

We asked AI models to build the Eiffel Tower in 3D, offline. Two of them added all 72 scientists' names to it, in order, from memory We did not expect this. Every AA-3D model got the same brief of about 200 words, on the tower's shape, lattice, platforms and lighting. It never mentions the names. Gustave Eiffel put the names of 72 French scientists, engineers, and mathematicians under the tower's first platform, 18 on each side. Claude Opus 5.5 and Grok 4.7 wrote all 72, with each side in the real order. Claude Opus 5.5 also put each set on the side that faces the same way in Paris. Each model built its scene as an agent in a sandbox with no internet access. In both runs, the names first appear in code the model wrote, never in a file or tool output it read. Congratulations to , Elon Musk and Anthropic on two seriously knowledgeable models

We asked AI models to build the Eiffel Tower in 3D, offline. Two of them added all 72 scientists' names to it, in order, from memory We did not expect this. Every AA-3D model got the same brief of about 200 words, on the tower's shape, lattice, platforms and lighting. It never mentions the names. Gustave Eiffel put the names of 72 French scientists, engineers, and mathematicians under the tower's first platform, 18 on each side. Claude Opus 5.5 and Grok 4.7 wrote all 72, with each side in the real order. Claude Opus 5.5 also put each set on the side that faces the same way in Paris. Each model built its scene as an agent in a sandbox with no internet access. In both runs, the names first appear in code the model wrote, never in a file or tool output it read. Congratulations to , Elon Musk and Anthropic on two seriously knowledgeable models

Artificial Analysis

36,993 views • 29 days ago

Compare AI models side by side with the Artificial Analysis model comparison tool Select any model and compare them across: ➤ Artificial Analysis Intelligence Index scores ➤ Individual benchmark scores ➤ Artificial Analysis Capability Index scores including Finance & Accounting, Strategy & Ops, Legal, Healthcare & Medical, Engineering, and Economics ➤ Cost, broken down by input, output, and cached tokens ➤ Output speed and latency Explore the tool:

Compare AI models side by side with the Artificial Analysis model comparison tool Select any model and compare them across: ➤ Artificial Analysis Intelligence Index scores ➤ Individual benchmark scores ➤ Artificial Analysis Capability Index scores including Finance & Accounting, Strategy & Ops, Legal, Healthcare & Medical, Engineering, and Economics ➤ Cost, broken down by input, output, and cached tokens ➤ Output speed and latency Explore the tool:

Artificial Analysis

26,511 views • 1 month ago

Last week we launched Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency Watch our new video walkthrough to see how to build and run your own benchmark with Optima, and find the best model for your use case Start building your benchmark today at

Last week we launched Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency Watch our new video walkthrough to see how to build and run your own benchmark with Optima, and find the best model for your use case Start building your benchmark today at

Artificial Analysis

19,837 views • 2 months ago

Example generations from HappyHorse-1.0 compared to Dreamina Seedance 2.0, Kling 3.0 Pro, grok-video-imagine and PixVerse V6 (Text to Video with Audio): Prompt [1/4]: A hula hoop spinning on a kid's waist, gradually climbing to their chest, then dropping to knees, then clattering to the floor. They pick it up to try again.

Example generations from HappyHorse-1.0 compared to Dreamina Seedance 2.0, Kling 3.0 Pro, grok-video-imagine and PixVerse V6 (Text to Video with Audio): Prompt [1/4]: A hula hoop spinning on a kid's waist, gradually climbing to their chest, then dropping to knees, then clattering to the floor. They pick it up to try again.

Artificial Analysis

36,791 views • 7 months ago

By popular demand, Model Sets are now live! You can now save custom selections of models and apply them instantly across all charts.

By popular demand, Model Sets are now live! You can now save custom selections of models and apply them instantly across all charts.

Artificial Analysis

22,669 views • 4 months ago

Gemini 3.5 Flash is a step forward for Google on speed and agentic capabilities but comes at a trade-off of being higher cost than prior models We have measured up to ~280 output tokens/sec, placing it on the speed/intelligence Pareto frontier and well ahead of Gemini 3 Flash. It also shows a major uplift on agentic tasks, reaching ~1650 ELO on GDPVal-AA. The trade-off: cost is up ~5x versus Gemini 3 Flash, driven by higher token prices (3x higher than Gemini 3 Flash) and higher token usage. In this video, Declan Jackson, Member of Technical Staff at Artificial Analysis, breaks it down.

Gemini 3.5 Flash is a step forward for Google on speed and agentic capabilities but comes at a trade-off of being higher cost than prior models We have measured up to ~280 output tokens/sec, placing it on the speed/intelligence Pareto frontier and well ahead of Gemini 3 Flash. It also shows a major uplift on agentic tasks, reaching ~1650 ELO on GDPVal-AA. The trade-off: cost is up ~5x versus Gemini 3 Flash, driven by higher token prices (3x higher than Gemini 3 Flash) and higher token usage. In this video, Declan Jackson, Member of Technical Staff at Artificial Analysis, breaks it down.

Artificial Analysis

16,055 views • 5 months ago

We’re unveiling a new look for Artificial Analysis! We’ve come a long way since launching Artificial Analysis over 2 years ago. Today, we benchmark 400+ models, 50+ inference providers, and benchmark not only language models but also image, video, speech, music, hardware, and agents. Our mission to support the AI ecosystem with independent benchmarking remains the same, but our brand and website refresh is designed to better reflect how much we’ve grown and how much further we plan to go. A huge thank you to everyone who has been part of the Artificial Analysis community along the way: from developers choosing models and building agents, to labs, inference and hardware providers, and fellow independent researchers.

Artificial Analysis

18,208 views • 7 months ago

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

Artificial Analysis

13,214 views • 4 months ago

Overview of our recent launch of Coding Agent benchmarks on Artificial Analysis and our first Youtube Video! We walk through the performance, cost, token usage and speed differences across different coding agents. This includes looking at Opus 4.7 in Claude Code's leading performance and Composer 2.5's strong positioning on the Coding Agent Index / Cost Pareto frontier. We have also launched our YouTube channel! Come say hi and subscribe:

Overview of our recent launch of Coding Agent benchmarks on Artificial Analysis and our first Youtube Video! We walk through the performance, cost, token usage and speed differences across different coding agents. This includes looking at Opus 4.7 in Claude Code's leading performance and Composer 2.5's strong positioning on the Coding Agent Index / Cost Pareto frontier. We have also launched our YouTube channel! Come say hi and subscribe:

Artificial Analysis

10,702 views • 5 months ago

Announcing Artificial Analysis Video Arena - the first crowdsourced comparison for Text to Video models Text to Video models are accelerating rapidly and crossing quality thresholds every month. We created Video Arena to compare them using the only source of truth for visual media - human preference! Video Arena includes hundreds of videos from the leading video models, including: - Runway's Runway Gen 3 Alpha - Pika's Pika 1.5 - Luma's Dream Machine - MiniMax / Hailuo AI (MiniMax) - Kling AI's Kling 1.0 - Zhipu AI's CogVideoX-5B Voting is open now and we’ll be announcing the first leaderboard results within 24 hours. Any predictions? In the meantime, you can see your own ‘personal leaderboard’ of how you’ve ranked the video models after 30 votes. Link to the Artificial Analysis Video Arena in the below tweet! 👇

Announcing Artificial Analysis Video Arena - the first crowdsourced comparison for Text to Video models Text to Video models are accelerating rapidly and crossing quality thresholds every month. We created Video Arena to compare them using the only source of truth for visual media - human preference! Video Arena includes hundreds of videos from the leading video models, including: - Runway's Runway Gen 3 Alpha - Pika's Pika 1.5 - Luma's Dream Machine - MiniMax / Hailuo AI (MiniMax) - Kling AI's Kling 1.0 - Zhipu AI's CogVideoX-5B Voting is open now and we’ll be announcing the first leaderboard results within 24 hours. Any predictions? In the meantime, you can see your own ‘personal leaderboard’ of how you’ve ranked the video models after 30 votes. Link to the Artificial Analysis Video Arena in the below tweet! 👇

Artificial Analysis

27,108 views • 2 years ago

No more content to load