Same prompt for all 4 tests, 100% code generated animation. GPT 6.1 Sol vs GPT 6 Astra vs Sonnet 5.5 vs Opus 5.5 Despite the huge price difference between the final results, if this was a blind test, we would not even say that Sol and Astra are on... the same level as Sonnet and Opus. On price, Sol was much cheaper than all other models, and maybe even better than Astra quality wise. On quality alone, Opus and Sonnet were above (by a lot), including richer details, camera movements, effects, being more fluid and engaging overall. And if we consider Price and Quality, Opus is the overall winner, costing 12$ vs 18$ from Sonnet, finishing in 1h and 16m vs 1h and 36m.show more

Claude really cooks when it comes to art and design. My workflow is Claude Design first then Claude and Codex adversarial reviews for planning but codex for the build. What prompt did you use?

Your workflow seems to deliver amazing stuff! Do you have any posts showing it? The prompt was super simple, something on the lines of: I want you to create a fast paced 15 second animation fight using stickmans. It should have a Jedi and a Sith, focus on it being engaging. You are not allowed to copy anything from other folders

Oh nice! Wow. Wasn't sure if you had to tell it to use three.js or anything like that. Yeah here's an example of an older app I am rebuilding that I built years ago before the AI era haha

Pretty cool!! When are you launching it back?

Welllll my goal was end of September. Classic "this will take 2 minutes to update"...1 month later...

You can do it this month! Tag me when you are done!!

🫡🫡 let's go bro

That's basically what I've felt too: Opus and Sonnet 5.5 are amazing

I don’t see the point of Sonnet yet! Its output is worst and more expensive than Opus in all my tests! (Some people liked Sonnet better on this one though) Have you found a use for it?

I've been using Sonnet 5.5 quite a bit and it's actually working fairly well. I've managed to keep my usage down pretty far and it seems to perform fairly well. Overall I prefer Opus 5.5 as well though they do seem to both work really well.

Holy fuck the Anthropic 5.5 models are … something else entirely. Man, if we ever get opus 5.5 quality at the cost and efficacy of sol 6.1 … Anthropic I hope makes a major leap in efficiency soon without comprising model. Opus 4.7-5 was awful. But now it really is a harder call to make.

Yeah considering the quality, Opus 5.5 using the 20$ subscription is already a very good value. On API costs they still need to make it cheaper for sure!

The $20 Anthropic plan is the best deal right now at that price point. Serious improvement there. I really hope Anthropic adds some features to Claude desktop to make it as nice as ChatGPT has been the last few weeks.

Yes I feel the same! Codex is the best harness, but for coding tasks Claude with Opus 5.5 is just amazing now

Dude the sonnet one was lit!!

Did you like it more over Opus one?

I think I might yes

Yeah it cooked hard! The sword exchange part was really fast paced!!!

You pay for quality, you get quality. That's what I feel when I see this demo done by Opus.

Yeah! But I believe Sol still has its right place under the sun, trying another test now with a time budget, let’s see!

what was the technology used for drawing and animation?

Sorry but I had to ask AI to answer properly: GPT 6.1 Sol Sol defines poses—running, guarding, striking, ducking, jumping—and interpolates between timed keyframes. It uses inverse kinematics, which calculates elbows and knees from the desired hand and foot positions. The fight stays in one reactor-hall environment, with camera pans and zooms. Saber trails and character afterimages come from drawing earlier poses; sparks use calculated blade intersections, with a midpoint fallback. GSAP drives the animation clock, and HyperFrames exports the video. Source: fight.js⁠. GPT 6 Astra Astra uses a similar Canvas pipeline, but its choreography has a reusable exchange function: it creates anticipation, contact, a brief hold and recoil for both fighters. At contact, both blades aim toward a shared point, deliberately coordinating the clash. It also implements limb IK, saber trails, procedural particles and background parallax. One technical distinction: the final composition draws frames directly from HyperFrames’ hf-seek events. GSAP is listed as a dependency, but the final animation clock doesn’t use it. Source: duel.js⁠. Sonnet 5.5 Sonnet built the most distinct pipeline: a Python animation engine split into rigging, choreography, scene, effects and rendering modules. It coordinates saber contact by aiming the defender’s blade at the attacker’s blade. A separate story clock controls slow motion and brief impact freezes while keeping the output exactly 15 seconds. Unlike Sol and Astra’s continuous presentation, it explicitly cuts between camera shots. Skia draws the scene. Each output frame averages 10 time samples for motion blur, followed by bloom, grain, vignette and impact-related RGB separation. Frames render across multiple processes and feed into FFmpeg. Sources: Opus 5.5 Opus also uses Canvas and HyperFrames, but implements a more elaborate effects pipeline: separate environment and luminous-effect layers, multi-scale bloom, smoke, embers, lightning and saber illumination. Its animation clock maps video time into story time to create deliberate speed ramps. Characters use keyframed skeletal rigs and IK; calculated blade proximity locates clash effects. Camera motion includes tracking, zoom, rotation and shake, with directional blur during fast pans. Source: duel.js⁠. For the narration: Sol and Astra built compact browser animation engines; Sonnet built a Python rendering pipeline; Opus built a browser engine with more extensive cinematic compositing.

can you also do Fable 5.1

Will do a full comparison when they release the next Fable!

whoa sonnet

Do you prefer it’s result over Opus ?

opus 5.5 with sol sub agents is the way people

Have you tried that? Which harness are you using to orchestrate it?

yes. i use and i had it build a custom sub agent extension.

Ohhh nice! I want to try pi, what is your opinion on it? Have you tweeked it a lot? Comparing to Claude Code CLI, is it better?

At this point how do you call OpenAI frontier lab? Reality is Astra was never a new class of model It was Sol class of model. And what we are getting as Sol is 300B distilled model that score good on benchmarks but actual quality is terribad. But is somewhat cheap.

I believe it is more of a question of perspective, when Astra came and there was only Opus 5, it was looking great. But Anthropic dropped Opus 5.5, and suddenly the bar was much higher for everyone

As I keep saying Astra was true Sol class model. But OpenAI got greedy and tried to upsell it as Mythos/Fable class at double the cost. If Astra was priced like Opus 5.5 then maybe then we can even compare them.

Yeah it makes sense

I wouldn’t say this determines which models are better.

Yes and I agree with you! This is only 1 vertical (animation), depending on what you want to do with it, the result completely changes. For my normal day to day tasks I am still using Codex (both Sol and Astra)

The difference is HUGE

Which one did you like the most?

I still have plenty of Codex usage quota left; if anyone needs to test anything, feel free to send a prompt.

Night and day difference

Did you like Sonnet or Opus better?

Not sure tbh both was good

Скиньте промт Затестим тоже

Can you send me the prompt ? To try it on argon now.

Sure here it is: Create me a 15 second Stickman animation based on a fight of a Sith vs a Jedi It should be super fast paced and engaging. It does not need sound effects. It should be 16-9. You are not authorized to copy programs or files from other folders.

thx you, here the result, argon extended gemini web

There is no question Claude does better on vague prompts. This is a given. What happens when you hand all the models a tight prompt and design spec, art board, etc?

Usually when you give them very specific instructions they will have a closer result. But this is something I think a lot now, the models became so advanced, that sometimes it is better to be vague on details you are not 100% sure that you want. That or let the model do a full interview on you to close all the gaps.

See this is what I mean though. What if you wanted some much simpler. Then clauses automatic flare - too much. Some people will have maximal ideas/vision. Others minimal.

What is the prompt?

Thank you

My pleasure

Dramaturgy skills of Opus 5.5 are mindblowing.

Yeah its results were way better! With sound effects it could become something really nice!

Anthropic fucking COOKING.

Totally!! They were in a bad position for some time with Opus 5…but now…

It's only animation though. Very limited application in the real world.

Yes true, and to be able to produce such outputs, shows how advanced the models are currently on delivering a complete task end to end

No doubt that these animations are amazing in the original sense of the word and provide yet another proof of how revolutionary this technology is. It's wild to think that we're only 3 or 4 years into it. Imagine what it'll be like in 5 or 10 years.

I believe the future now lies in this along with Robotics! We will see kids programming robots with natural language, what do you see?

I think it's fun to speculate but I honestly have no idea what the world will look like in 10 years: whether there'll be robots everywhere, many people getting minimum income and living lives of enforced leisure, agriculture transformed by robots as an example, driverless cars as the norm; or whether it'll be a bit like it is today, with just more AI, but basically in business workflows more than anything else, and robots mostly in factories.

Hahaha wtf 😂 I made simial animation on my first computer 30 years ago 😂

For real? Do you still have it?

Sadly no 😕 and ofcourse the quality wasn't anywhere near these but stick figures fighting with light sabers and the moves look so similiar. No special effects either. I was less than 10 year old star wars fan and computers were 486's back then 😂

You are absolutely right, but if GPT was prompted a little bit differently and allowed it to work for longer, perhaps it would have a better result. I know that's not the point, but clearly Codex works for a lot less time than Claude.

Yeah I was intrigued by that as well...so I just tested it! I gave GPT 6.1 Sol 2 hours to complete the task, will make another post about the result!

Looks like Opus dominates the EV curve when you weight cost against output quality

Yes totally!! But people suggested another test that I want to give a try. Submit the prompt with a time limit for all models and submit the same with a budget limit as well!

Makes sense, testing with time limits and budget caps should reveal clear trade offs.

sol이랑 Astra에게도 동일한 시간만큼 추가 작업을 시켰다면? 아니면 프롬프트를 제대로 작성해서 동일한 작업을 시켰다면? 시간과 비용을 덜쓰고 같은 효과를 바라고 비교한다는 자체가 멍청한거 같은데?

I made a post about that test. The model ended up onl doing small adjustments and did not get the same quality at all as Opus.

Utilizaste sólo la app de Claude o lo conectaste con runway o alguna otra aplicación o plugging?

Both GPT tests were done on Codex CLI and both Anthropic tests were on Claude CLI. Code only! No other text to video models or applications were used!

A blind test is the fairest test. No hype, just outputs.

Agree!!

Drop the prompt

you should give them all exactly 2 hours to see what each one come up with

Hmmm good idea, will try that !!!

How about giving all of them a $10 budget and let them run. I think that would have shifted the tables quite a bit

Yes I think this will be my next test! Either a $ budget or time limit and see who can make the better end result! It makes sense on the real world at least, when someone handles you a task they will either give a deadline or a budget

Great, be interesting to see. Yes, I think that would be closer to real world use and in a sense more fair to the models ( I suspect Sol 6.1 will shine here)

Yeah! Since they made Sol faster it became easier to work with it (the 20 tok/sec on launch day was horrible)

no sound?

Yes! I made it without sound because I will use those tests on my next YouTube video, which will have my voice over it.

Exactly you nailed it, this is the type of disappointment I currently face and OpenAI are trying to keep customers by offering them banked resets but it doesn't matter when quality is lacking

Thanks a lot!!

OK openai you know what to do now. DISTILL FROM OPUS!!!

Hahahahaha we know at least the new Chinese models will be monsters distilled from these guys

yooo sonnet out there being fire.

Wow....good benchmark, the style reminds me of "Stick Death" ; Thank you for sharing. Makes you wonder about the next iteration of Fable. 🤯