Luckey Faraday's banner
Luckey Faraday's profile picture

Luckey Faraday

@luckeyfaraday • 2,764 subscribers

Exploring the AI frontier: models, agents, benchmarks & applied experiments. I test, compare and build. (kinda got DMCA'd by two companies we don't talk about)

Shorts

Took the Mario test one step further. GPT Astra, GPT-6.1 Sol, Sonnet 5.5 and Opus 5.5, all on the exact same prompt. Really interesting results here, especially considering the massive spread in cost. Which one wins? OpenAI or Anthropic?

Took the Mario test one step further. GPT Astra, GPT-6.1 Sol, Sonnet 5.5 and Opus 5.5, all on the exact same prompt. Really interesting results here, especially considering the massive spread in cost. Which one wins? OpenAI or Anthropic?

171,539 views

People are saying Opus 5.5 might have been nerfed. So I reran the exact same Mario and Mickey Mouse tests with September 30th Opus 5.5 vs. October 3rd Opus 5.5. Same prompts, just three days apart. What do you think? Is Opus 5.5 nerfed?

People are saying Opus 5.5 might have been nerfed. So I reran the exact same Mario and Mickey Mouse tests with September 30th Opus 5.5 vs. October 3rd Opus 5.5. Same prompts, just three days apart. What do you think? Is Opus 5.5 nerfed?

95,374 views

GPT Astra vs. GPT-6.1 Sol and every previous Sol generation on the Mario test. Same prompt, same test. Astra still wins, finishing in just 15m 37s for $2.96. GPT-6.1 Sol took 34m 38s, more than twice as long, but cost just $0.64 and delivered the best result of any Sol model. Astra is still ahead, but 6.1 Sol is getting surprisingly close for a fraction of the price.

GPT Astra vs. GPT-6.1 Sol and every previous Sol generation on the Mario test. Same prompt, same test. Astra still wins, finishing in just 15m 37s for $2.96. GPT-6.1 Sol took 34m 38s, more than twice as long, but cost just $0.64 and delivered the best result of any Sol model. Astra is still ahead, but 6.1 Sol is getting surprisingly close for a fraction of the price.

74,367 views

Videos