Artificial Analysis's banner
Artificial Analysis's profile picture

Artificial Analysis

@ArtificialAnlys • 165,273 subscribers

Independent analysis of AI

Shorts

Same robot, same task, two models with very different writing styles ➤ On the left, Claude Opus 5.5 dabs out "hello world" dot by dot: 3,270 arm moves over 34 minutes ➤ On the right, GPT-6 Astra writes in continuous strokes: 514 moves in 13 minutes From AA-Robotics v0.1, seen from the model’s view See more examples here:

Same robot, same task, two models with very different writing styles ➤ On the left, Claude Opus 5.5 dabs out "hello world" dot by dot: 3,270 arm moves over 34 minutes ➤ On the right, GPT-6 Astra writes in continuous strokes: 514 moves in 13 minutes From AA-Robotics v0.1, seen from the model’s view See more examples here:

59,251 görüntüleme

Fish Audio Drama 3 is the #1 open weights model on both Artificial Analysis Speech Arenas, and #1 of all models on Pronunciation Robustness Fish Audio has released Drama 3. We evaluated it across both Speech Arenas and our Pronunciation Robustness benchmark. Key takeaways: ➤ Text to Speech, Provider Voice: Drama 3 scores an Elo of 1,278 across 1,512 arena appearances, ranking #6 overall and #1 among open weights models, 38 points clear of its own Drama 3 Preview at 1,240 and 50 ahead of BreezeBlue's Breeze TTS 2 at 1,228 ➤ Text to Speech, Controlled Voice: Drama 3 scores 1,085 across 1,958 appearances, ranking #12 overall and #1 among open weights models, ahead of Ant Group's Ming-3.0-tts at 1,068 and its own Preview at 1,063 ➤ Pronunciation Robustness: Drama 3 scores 94.6% across 2,157 rated spans, ranking #1, ahead of Ming-3.0-tts and Drama 3 Preview, both at 93.2% ➤ Speed: Drama 3 generates 66.6 characters per second, ahead of its own Drama 3 Preview at 60.7 and Murf's Speech Gen 2 at 63.0, and behind ElevenLabs' Eleven v4 at 70.0 and v3 Conversational at 66.8 ➤ Pricing: Drama 3 is priced at $15/1M characters. Every model ranked above it on Provider Voice costs more: Gemini 3.8 Flash TTS at $16.5/1M, Qwen-Audio-3.1-TTS-Plus at $19.3/1M, Eleven v4 Turbo at $40/1M, Sonic 3.6 at $49/1M and Eleven v4 at $80/1M ➤ Open weights: released under the Fish Audio Community License. See more details and listen to samples below ⬇️

Fish Audio Drama 3 is the #1 open weights model on both Artificial Analysis Speech Arenas, and #1 of all models on Pronunciation Robustness Fish Audio has released Drama 3. We evaluated it across both Speech Arenas and our Pronunciation Robustness benchmark. Key takeaways: ➤ Text to Speech, Provider Voice: Drama 3 scores an Elo of 1,278 across 1,512 arena appearances, ranking #6 overall and #1 among open weights models, 38 points clear of its own Drama 3 Preview at 1,240 and 50 ahead of BreezeBlue's Breeze TTS 2 at 1,228 ➤ Text to Speech, Controlled Voice: Drama 3 scores 1,085 across 1,958 appearances, ranking #12 overall and #1 among open weights models, ahead of Ant Group's Ming-3.0-tts at 1,068 and its own Preview at 1,063 ➤ Pronunciation Robustness: Drama 3 scores 94.6% across 2,157 rated spans, ranking #1, ahead of Ming-3.0-tts and Drama 3 Preview, both at 93.2% ➤ Speed: Drama 3 generates 66.6 characters per second, ahead of its own Drama 3 Preview at 60.7 and Murf's Speech Gen 2 at 63.0, and behind ElevenLabs' Eleven v4 at 70.0 and v3 Conversational at 66.8 ➤ Pricing: Drama 3 is priced at $15/1M characters. Every model ranked above it on Provider Voice costs more: Gemini 3.8 Flash TTS at $16.5/1M, Qwen-Audio-3.1-TTS-Plus at $19.3/1M, Eleven v4 Turbo at $40/1M, Sonic 3.6 at $49/1M and Eleven v4 at $80/1M ➤ Open weights: released under the Fish Audio Community License. See more details and listen to samples below ⬇️

29,210 görüntüleme

We gave AI models one prompt: build an interactive black hole, physically informed, with the presence of Gargantua from Interstellar.

We gave AI models one prompt: build an interactive black hole, physically informed, with the presence of Gargantua from Interstellar.

40,948 görüntüleme

You can test GPT-6 Astra on your own custom benchmarks in Optima Head to to start creating and running your own custom benchmarks today

You can test GPT-6 Astra on your own custom benchmarks in Optima Head to to start creating and running your own custom benchmarks today

18,129 görüntüleme

You can now benchmark Qwen3.8 27B with Optima. See how a model that can run on your laptop compares on performance, cost and speed for your custom use cases We're also offering credits and free advisory from an Artificial Analysis team member to enterprises looking to build benchmarks for their use cases. Let us know if you’re interested: Build and run your custom benchmarks today at

You can now benchmark Qwen3.8 27B with Optima. See how a model that can run on your laptop compares on performance, cost and speed for your custom use cases We're also offering credits and free advisory from an Artificial Analysis team member to enterprises looking to build benchmarks for their use cases. Let us know if you’re interested: Build and run your custom benchmarks today at

15,764 görüntüleme

Want to generate images from the world's best models like Nano Banana or GPT Image side by side? Now you can with Image Lab 🚀 Run a single prompt across up to 25 models, with up to 20 images from each, and see results in seconds. You've seen our leaderboards. Now generate and evaluate the models yourself. Link in thread 🧵👇

Want to generate images from the world's best models like Nano Banana or GPT Image side by side? Now you can with Image Lab 🚀 Run a single prompt across up to 25 models, with up to 20 images from each, and see results in seconds. You've seen our leaderboards. Now generate and evaluate the models yourself. Link in thread 🧵👇

12,250 görüntüleme

Videos

ArtificialAnlys's profile picture

Welcome to the era of embodied AI. We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today. We find that frontier language models in coding agent harnesses have recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly. GPT-6 Astra places first in AA-Robotics v0.1, followed by GPT-6.1 Sol, Gemini 4 Argon and Claude Opus 5.5. AA-Robotics v0.1 details: ➤ Our first eight tasks test a range of capabilities: control and dexterity (precise grasping and object handling), spatial reasoning (how objects interact with one another) and adaptability (how models respond to the unexpected) ➤ Models get only brief instructions: how to connect to the robot arm (via a simple HTTP API), the arm's name and how to submit an attempt. They can view the scene through four cameras and write and run code to plan and send movements ➤ Once a model submits an attempt, its end state is graded against the task's pass criteria. A pass scores 100; an incomplete attempt earns partial credit. All attempts are zero-shot Key findings: ➤ The GPT family tops the leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. Claude Opus 5.5 (49) and Claude Fable 5.1 (46) trail ➤ Today’s models are incredibly slow at our tasks, taking ~30 minutes on average to complete each tabletop task ➤ Cost varies widely: GPT-6.1 Sol is the cheapest at $1.93 per task on average; GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much ➤ Compared with a human, today's best-performing frontier language models are ~100x slower and ~10x more expensive More results, clips, and how AA-Robotics v0.1 works follow in the thread below.

Artificial Analysis

38,349 görüntüleme • 15 gün önce

ArtificialAnlys's profile picture

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

133,823 görüntüleme • 2 ay önce

Daha fazla içerik yok.