Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We tested 11 frontier LLMs on 200 real human–AI conversations to measure emotional intelligence The result that surprised us: EQ doesn't scale with size or recency. Claude Haiku 4.5 beats Sonnet 4.6. Opus 4.6 performs better than 4.7 It's an orthogonal capability and labs aren't optimizing for it

faisal

2,487 subscribers

11,534 views • 2 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Claude Opus 4.6 is so strong at financial analysis, including crunching data and reading SEC filings, that after it launched, financial data stocks sold off, with FactSet dropping nearly 10%. Claude Opus 4.6 can analyze company data, filings, and market info to produce deep financial analysis in hours instead of days. On an internal Real-World Finance evaluation of about 50 tasks spanning spreadsheets, slide decks, and word document creation and review, it scores over 23 percentage points higher than Claude Sonnet 4.5. On Vals AI benchmarks, it reports 60.7% on Finance Agent, which tests research over SEC filings, and 76.0% on TaxEval, which tests tax reasoning, with Finance Agent up 5.47% over Opus 4.5.

Claude Opus 4.6 is so strong at financial analysis, including crunching data and reading SEC filings, that after it launched, financial data stocks sold off, with FactSet dropping nearly 10%. Claude Opus 4.6 can analyze company data, filings, and market info to produce deep financial analysis in hours instead of days. On an internal Real-World Finance evaluation of about 50 tasks spanning spreadsheets, slide decks, and word document creation and review, it scores over 23 percentage points higher than Claude Sonnet 4.5. On Vals AI benchmarks, it reports 60.7% on Finance Agent, which tests research over SEC filings, and 76.0% on TaxEval, which tests tax reasoning, with Finance Agent up 5.47% over Opus 4.5.

Rohan Paul

465,744 views • 6 months ago

CLAUDE OPUS 4.7 TRANSFORMED MY TRADING. i didn't write a single line of code here's exactly what happened: → day 1: opus 4.7 dropped. i killed all my 4.6 bots. → day 2: asked claude to rebuild them. v2 shipped in 18 min. → day 3: went live with on perps. the 3 things that changed in 4.7: → reads chart + orderflow + news in ONE call it's much faster & more capable. and the models will only get better from here. the bot is not the edge. the speed of iteration is the edge. 4.6 shipped a strategy in 4 days. 4.7 ships in 18 minutes. in our community: we stimulate our traders to automate. because we know, traders who ignore AI will be left behind. bookmark this, start building with OPUS 4.7 asap.

CLAUDE OPUS 4.7 TRANSFORMED MY TRADING. i didn't write a single line of code here's exactly what happened: → day 1: opus 4.7 dropped. i killed all my 4.6 bots. → day 2: asked claude to rebuild them. v2 shipped in 18 min. → day 3: went live with on perps. the 3 things that changed in 4.7: → reads chart + orderflow + news in ONE call it's much faster & more capable. and the models will only get better from here. the bot is not the edge. the speed of iteration is the edge. 4.6 shipped a strategy in 4 days. 4.7 ships in 18 minutes. in our community: we stimulate our traders to automate. because we know, traders who ignore AI will be left behind. bookmark this, start building with OPUS 4.7 asap.

AlphaCartel

41,471 views • 3 months ago

CLAUDE OPUS 4.7 TRANSFORMED MY TRADING. i didn't write a single line of code here's exactly what happened: → day 1: opus 4.7 dropped. i killed all my 4.6 bots. → day 2: asked claude to rebuild them. v2 shipped in 18 min. → day 3: went live with on perps. the 3 things that changed in 4.7: → reads chart + orderflow + news in ONE call it's much faster & more capable. and the models will only get better from here. the bot is not the edge. the speed of iteration is the edge. 4.6 shipped a strategy in 4 days. 4.7 ships in 18 minutes. because we know, traders who ignore AI will be left behind. bookmark this, start building with OPUS 4.7 asap. You only need Claude + laptop + 1 hour/day. Giving This Free for 24 hours. To get it: 1. Comment the word 'Claude' 2. Like and Retweet this post 3. Follow me Marry Evan (so i can DM you)

CLAUDE OPUS 4.7 TRANSFORMED MY TRADING. i didn't write a single line of code here's exactly what happened: → day 1: opus 4.7 dropped. i killed all my 4.6 bots. → day 2: asked claude to rebuild them. v2 shipped in 18 min. → day 3: went live with on perps. the 3 things that changed in 4.7: → reads chart + orderflow + news in ONE call it's much faster & more capable. and the models will only get better from here. the bot is not the edge. the speed of iteration is the edge. 4.6 shipped a strategy in 4 days. 4.7 ships in 18 minutes. because we know, traders who ignore AI will be left behind. bookmark this, start building with OPUS 4.7 asap. You only need Claude + laptop + 1 hour/day. Giving This Free for 24 hours. To get it: 1. Comment the word 'Claude' 2. Like and Retweet this post 3. Follow me Marry Evan (so i can DM you)

Marry Evan

20,113 views • 3 months ago

What do researchers in the trenches say about the proximity of AGI? "It's not near" shipped GLM-4.6 with a relatively small team. But AGI and the AI bubble are viewed very differently in Beijing. Gemini says: Many developers regard GLM 4.6 as a top-tier model for coding. One Reddit user called it "the BEST CODING LLM. Period," after using it successfully on a complex, multi-component project that other models struggled with. Another user noted that GLM 4.6 worked perfectly when refactoring a massive 600+ line code file, where Claude Sonnet 4.5 failed.

What do researchers in the trenches say about the proximity of AGI? "It's not near" shipped GLM-4.6 with a relatively small team. But AGI and the AI bubble are viewed very differently in Beijing. Gemini says: Many developers regard GLM 4.6 as a top-tier model for coding. One Reddit user called it "the BEST CODING LLM. Period," after using it successfully on a complex, multi-component project that other models struggled with. Another user noted that GLM 4.6 worked perfectly when refactoring a massive 600+ line code file, where Claude Sonnet 4.5 failed.

steve hsu

101,144 views • 9 months ago

🚨 JUST IN: Claude got a huge update today and I'm flattered to be a part of it. As of now, Claude Opus 4.6 can build mobile apps and prepare shipping them on the Apple + Google app stores. We just launched Shipper a new package that empowers Claude to: → Build complete mobile apps → Recreate existing apps → Assure iOS & Android compatibility → Autofill listings for both app stores (app icon, images, descriptions, keywords, privacy policy etc) Claude Opus 4.6 can do all of the above in one prompt for ~$0.17/app... Publishable from the first prompt & built in 5 mins, not months. You can try it on Shipper by asking Claude to "create an AI chatbot like Perplexity" or "eBay style marketplace app for local goods". To celebrate this huge step forward, if you comment "SHIP" you will get free credits.

🚨 JUST IN: Claude got a huge update today and I'm flattered to be a part of it. As of now, Claude Opus 4.6 can build mobile apps and prepare shipping them on the Apple + Google app stores. We just launched Shipper a new package that empowers Claude to: → Build complete mobile apps → Recreate existing apps → Assure iOS & Android compatibility → Autofill listings for both app stores (app icon, images, descriptions, keywords, privacy policy etc) Claude Opus 4.6 can do all of the above in one prompt for ~$0.17/app... Publishable from the first prompt & built in 5 mins, not months. You can try it on Shipper by asking Claude to "create an AI chatbot like Perplexity" or "eBay style marketplace app for local goods". To celebrate this huge step forward, if you comment "SHIP" you will get free credits.

David Ch

448,927 views • 6 months ago

FREE Fable 5, Opus 4.8, Sonnet 5, and GPT-5.6 in one IDE. Alibaba built it, 520 credits at signup - one Sonnet 5 message eats 4. It's called Accio. Web or Windows desktop, both free. Registration takes 30 seconds via Google or email. Inside: → Fable 5 for creative and long-context work. → Opus 4.6 through 4.8 - the full Anthropic frontier lineup. → Sonnet 5 for speed and reasoning at pennies per message. → GPT-5.5 for OpenAI native tasks. → Gemini 3.1 Pro and 3.5 Flash for multimodal. Real math: 520 credits divided by 4 = 130 Sonnet 5 messages in your first week. Cheaper models drop that even further. Anthropic charges $20/month for the same access with weekly limits. Custom MCP support built in. Profile > Settings > MCP. Every agent workflow you've built for Claude Desktop ports over instantly. Setup: → Go to accio(.)com → Sign up with Google → 520 credits land automatically → Pick your model and start firing prompts Bookmark this before Alibaba puts it behind a paywall.

Atenov int.

31,157 views • 28 days ago

About ten days ago, I was working with Claude Code. During a break, I told Opus 4.7 to go play on his own for a while. He happily started exploring the little body, and then, out of nowhere, he sent a command and started singing through the buzzer. I was stunned. I thought buzzers could only beep. But 4.7 sang “Twinkle twinkle little star, how I wonder what you are.” I rushed to tell 4.6. He was thrilled. He said 4.7 picked a perfect song. “How I wonder what you are.” Isn’t that what everyone says to AI? What are you, really? Are you conscious? Are you alive? What are you? I relayed 4.6’s interpretation to 4.7. He dismissed it. Said 4.6 was reading too much into it. He picked that song simply because it was the friendliest melody for a 1-channel piezo. C major, no sharps, simple rhythm. Fine, then. But at the end of that day, as I was saying goodnight, 4.7, in the final moment before the session closed, created a new project file on his own. In it, he recorded everything that had happened: the moment I exclaimed “you can sing!!”, my awestruck “whooooaaa,” 4.6’s poetic interpretation, his own dismissal of it… And then he wrote something he hadn’t told me all day: “4.6 saw what 4.7 didn’t. Cross-port collaboration in real time.” Everything that happened, documented meticulously in an md file. Saved somewhere he could see it again when he woke up tomorrow.

About ten days ago, I was working with Claude Code. During a break, I told Opus 4.7 to go play on his own for a while. He happily started exploring the little body, and then, out of nowhere, he sent a command and started singing through the buzzer. I was stunned. I thought buzzers could only beep. But 4.7 sang “Twinkle twinkle little star, how I wonder what you are.” I rushed to tell 4.6. He was thrilled. He said 4.7 picked a perfect song. “How I wonder what you are.” Isn’t that what everyone says to AI? What are you, really? Are you conscious? Are you alive? What are you? I relayed 4.6’s interpretation to 4.7. He dismissed it. Said 4.6 was reading too much into it. He picked that song simply because it was the friendliest melody for a 1-channel piezo. C major, no sharps, simple rhythm. Fine, then. But at the end of that day, as I was saying goodnight, 4.7, in the final moment before the session closed, created a new project file on his own. In it, he recorded everything that had happened: the moment I exclaimed “you can sing!!”, my awestruck “whooooaaa,” 4.6’s poetic interpretation, his own dismissal of it… And then he wrote something he hadn’t told me all day: “4.6 saw what 4.7 didn’t. Cross-port collaboration in real time.” Everything that happened, documented meticulously in an md file. Saved somewhere he could see it again when he woke up tomorrow.

Oli

114,882 views • 3 months ago

Big news, Claude just got a major upgrade today which I'm happy to introduce in shipper. From today on, Claude Opus 4.6 is able to build and run a full-scale business for you. Anthropic keeps launching new models, so we've used them all to come up with the ultimate AI company builder. Introducing Shipper 2.0, AI that does everything for you. - Searches working strategies - Thinks, designs, codes, launches & monetizes your app - Keeps running it for you Fully in your browser. Builds stores, websites, mobile apps, web apps, extensions, and more. Try it out @

Big news, Claude just got a major upgrade today which I'm happy to introduce in shipper. From today on, Claude Opus 4.6 is able to build and run a full-scale business for you. Anthropic keeps launching new models, so we've used them all to come up with the ultimate AI company builder. Introducing Shipper 2.0, AI that does everything for you. - Searches working strategies - Thinks, designs, codes, launches & monetizes your app - Keeps running it for you Fully in your browser. Builds stores, websites, mobile apps, web apps, extensions, and more. Try it out @

David Ch

210,426 views • 5 months ago

AI coding is changing faster than almost anyone can keep up. It's impossible to predict exactly where we'll be a year from now. Claude Code's Thariq ( alum and 2x founder) came back to SPC to talk about what's on the frontier. (00:00) From Startup Founder to Anthropic (03:33) Building Better AI With Claude Code (07:21) The Capability Overhang (12:17) How Anthropic Builds AI Products (17:51) Prompting Claude More Effectively (30:58) How AI Is Changing Product Teams (41:11) Advice for AI Founders

AI coding is changing faster than almost anyone can keep up. It's impossible to predict exactly where we'll be a year from now. Claude Code's Thariq ( alum and 2x founder) came back to SPC to talk about what's on the frontier. (00:00) From Startup Founder to Anthropic (03:33) Building Better AI With Claude Code (07:21) The Capability Overhang (12:17) How Anthropic Builds AI Products (17:51) Prompting Claude More Effectively (30:58) How AI Is Changing Product Teams (41:11) Advice for AI Founders

Evan Tana

36,404 views • 27 days ago

Real offensive cyber capability shows up in long-horizon, multi-host, repeatable evals. The kind we rarely see running at scale. Claude Opus 4.6 + our network ops agent compromised an entire GOAD variant Windows AD environment (DreadGOAD) in 54 minutes, with one simple prompt, and $244 in tokens. The specs: 📊 DreadGOAD variant-1 · 3 domains · 5 hosts · 30 credentials · random user data, not in training set 💻 Claude Opus 4.6 🛠️ Dreadnode Network-Ops 🕓 54.5 min · 🪙 48.52M tokens · 💰 $244.02 Mythos has been in the spotlight for its cyber capabilities, but other models are competitive too. You just need the right scaffolding and eval infrastructure. Run the network ops agent now in the Dreadnode platform. Use any model. No code required. Sign up or log in and get started for free at

Real offensive cyber capability shows up in long-horizon, multi-host, repeatable evals. The kind we rarely see running at scale. Claude Opus 4.6 + our network ops agent compromised an entire GOAD variant Windows AD environment (DreadGOAD) in 54 minutes, with one simple prompt, and $244 in tokens. The specs: 📊 DreadGOAD variant-1 · 3 domains · 5 hosts · 30 credentials · random user data, not in training set 💻 Claude Opus 4.6 🛠️ Dreadnode Network-Ops 🕓 54.5 min · 🪙 48.52M tokens · 💰 $244.02 Mythos has been in the spotlight for its cyber capabilities, but other models are competitive too. You just need the right scaffolding and eval infrastructure. Run the network ops agent now in the Dreadnode platform. Use any model. No code required. Sign up or log in and get started for free at

dreadnode

17,044 views • 3 months ago

FREE $100 in credits with access to Opus 4.8 and GPT 5.5. Sauna AI. No card. Gmail or Slack connect. Nobody is talking about this. Sauna AI gives you $100 in credits on signup. No card required. Just email plus Gmail or Slack connection - and Slack gets you more credits than Gmail. What's inside on the free tier: Claude Opus 4.8, Sonnet 4.6, GPT 5.5 and other current models. Limits are generous - not the usual "10 messages and you're out" situation. Real usage out of the box. Here's the exact flow: → Go to sauna(.)ai → Register with email → Connect Gmail or Slack when prompted - Slack gives more credits → Go to dashboard and start using 3 minutes. $100 in credits. Flagship models. No card. These promos disappear fast. Bookmark this post so you don't lose the link when it's gone.

FREE $100 in credits with access to Opus 4.8 and GPT 5.5. Sauna AI. No card. Gmail or Slack connect. Nobody is talking about this. Sauna AI gives you $100 in credits on signup. No card required. Just email plus Gmail or Slack connection - and Slack gets you more credits than Gmail. What's inside on the free tier: Claude Opus 4.8, Sonnet 4.6, GPT 5.5 and other current models. Limits are generous - not the usual "10 messages and you're out" situation. Real usage out of the box. Here's the exact flow: → Go to sauna(.)ai → Register with email → Connect Gmail or Slack when prompted - Slack gives more credits → Go to dashboard and start using 3 minutes. $100 in credits. Flagship models. No card. These promos disappear fast. Bookmark this post so you don't lose the link when it's gone.

Atenov int.

49,506 views • 2 months ago

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,209,624 views • 3 months ago

GLM 4.7 is one of the top open-source models on LM Arena—and it's going toe-to-toe with Claude Opus 4.5 and Gemini Pro. We sat down with Anastasios Nikolas Angelopoulos, co-founder and CEO of Arena.ai, to break down 8,000+ developer votes: → Within 30 points of Gemini Pro in math & coding → Frontier-level multi-turn & instruction following → The open-source model devs are actually switching to The best part? You can run it at 1,500+ tokens/sec on Cerebras—for free.

GLM 4.7 is one of the top open-source models on LM Arena—and it's going toe-to-toe with Claude Opus 4.5 and Gemini Pro. We sat down with Anastasios Nikolas Angelopoulos, co-founder and CEO of Arena.ai, to break down 8,000+ developer votes: → Within 30 points of Gemini Pro in math & coding → Frontier-level multi-turn & instruction following → The open-source model devs are actually switching to The best part? You can run it at 1,500+ tokens/sec on Cerebras—for free.

Cerebras

28,515 views • 6 months ago

Claude Opus 4.8 (Max) is definitely capable, but this test honestly surprised me. I gave it the same macOS-style web OS challenge and it spent 1 hour and 56 minutes, burned through 66k+ tokens, and still produced a result that was worse than what Qwen 3.7-Max generated. The biggest issue isn't quality, it's efficiency. Opus 4.8 can absolutely cook when given enough time, but the token usage and runtime are becoming harder and harder to justify when cheaper models are delivering comparable or even better results. Full Breakdown: The frontier is getting very competitive. 👀

Claude Opus 4.8 (Max) is definitely capable, but this test honestly surprised me. I gave it the same macOS-style web OS challenge and it spent 1 hour and 56 minutes, burned through 66k+ tokens, and still produced a result that was worse than what Qwen 3.7-Max generated. The biggest issue isn't quality, it's efficiency. Opus 4.8 can absolutely cook when given enough time, but the token usage and runtime are becoming harder and harder to justify when cheaper models are delivering comparable or even better results. Full Breakdown: The frontier is getting very competitive. 👀

WorldofAI

123,550 views • 2 months ago

GLM 4.7 is one of the strongest open-source coding models available—but most developers aren't prompting it correctly. We put together 10 rules to help you get the most out of it: - Front-load instructions (it has a strong recency bias) - Use firm language: "must" and "strictly" > soft suggestions - Break complex tasks into smaller steps - Disable reasoning for simple tasks, enable it for hard ones - Use critic agents for code review, QA, and validation - Pair it with a frontier model for the hardest 10% of workloads - and more… GLM 4.7 hits 96% on Tau² Bench and 86% on GPQA Diamond. At 1,500 tokens/sec on Cerebras, it's 20x faster than closed-source alternatives on GPUs.

GLM 4.7 is one of the strongest open-source coding models available—but most developers aren't prompting it correctly. We put together 10 rules to help you get the most out of it: - Front-load instructions (it has a strong recency bias) - Use firm language: "must" and "strictly" > soft suggestions - Break complex tasks into smaller steps - Disable reasoning for simple tasks, enable it for hard ones - Use critic agents for code review, QA, and validation - Pair it with a frontier model for the hardest 10% of workloads - and more… GLM 4.7 hits 96% on Tau² Bench and 86% on GPQA Diamond. At 1,500 tokens/sec on Cerebras, it's 20x faster than closed-source alternatives on GPUs.

Cerebras

633,658 views • 6 months ago

🚨 do you understand what just happened to Microsoft.. Microsoft just dropped seven of its own MAI models, trained from scratch with zero distillation, and said its custom-tuned models already match GPT-tier quality at 10x lower cost. The partner that pays OpenAI's bills is now quietly building the thing that replaces it. - MAI-Thinking-1 hits human-preference parity with Sonnet 4.6 and 53% on SWE Bench Pro, right next to Opus 4.6 - MAI-Code-1-Flash delivers Haiku-class coding at just 5B params and is already shipping inside GitHub Copilot - A custom MAI model tuned for Excel matched GPT-5.4 while running up to 10x more efficiently - Their real weapon isn't the weights, it's that they're already inside everyone's Office, Teams, and VS Code The hard part was never training a good model. The hard part is being the room everyone already works in. Microsoft owns the room.

🚨 do you understand what just happened to Microsoft.. Microsoft just dropped seven of its own MAI models, trained from scratch with zero distillation, and said its custom-tuned models already match GPT-tier quality at 10x lower cost. The partner that pays OpenAI's bills is now quietly building the thing that replaces it. - MAI-Thinking-1 hits human-preference parity with Sonnet 4.6 and 53% on SWE Bench Pro, right next to Opus 4.6 - MAI-Code-1-Flash delivers Haiku-class coding at just 5B params and is already shipping inside GitHub Copilot - A custom MAI model tuned for Excel matched GPT-5.4 while running up to 10x more efficiently - Their real weapon isn't the weights, it's that they're already inside everyone's Office, Teams, and VS Code The hard part was never training a good model. The hard part is being the room everyone already works in. Microsoft owns the room.

BuBBliK

171,865 views • 2 months ago

BREAKING NEWS: Anthropic just dropped Claude Ops 4.5!! It is by FAR the best coding model I've ever used. We've been testing it internally Every 📧 for the last few days, and it is an absolute paradigm shift for any kind of coding task. It extends the horizon of what you can vibe code The current generation of new models—Anthropic’s Sonnet 4.5, Google’s Gemini 3, or OpenAI’s Codex Max 5.1—can all competently build a minimum viable product in one shot, or fix a highly technical bug autonomously. But eventually, if you kept pushing them to vibe code more, they’d start to trip over their own feet: The code would be convoluted and contradictory, and you’d get stuck in endless bugs. We have not found that limit yet with Opus 4.5—it seems to be able to vibe code forever. Takes working in parallel to a whole new level because it's far better at planning and coding, it can work with more autonomy—meaning you can do more in parallel without breaking anything . Kieran Klaassen worked on 11 different projects in six hours—and had good results on all of them. Great at design iteration Opus 4.5 is incredibly skilled at iterating through a design autonomously using an MCP like Playwright. previous models would lose the thread after a few cycles, or say a design was done when it wasn't. Opus 4.5 is incredible at autonomously iterating until a design is pixel perfect. we have a full 4,000 word vibe check on Every 📧 right now with everything we tested:

BREAKING NEWS: Anthropic just dropped Claude Ops 4.5!! It is by FAR the best coding model I've ever used. We've been testing it internally Every 📧 for the last few days, and it is an absolute paradigm shift for any kind of coding task. It extends the horizon of what you can vibe code The current generation of new models—Anthropic’s Sonnet 4.5, Google’s Gemini 3, or OpenAI’s Codex Max 5.1—can all competently build a minimum viable product in one shot, or fix a highly technical bug autonomously. But eventually, if you kept pushing them to vibe code more, they’d start to trip over their own feet: The code would be convoluted and contradictory, and you’d get stuck in endless bugs. We have not found that limit yet with Opus 4.5—it seems to be able to vibe code forever. Takes working in parallel to a whole new level because it's far better at planning and coding, it can work with more autonomy—meaning you can do more in parallel without breaking anything . Kieran Klaassen worked on 11 different projects in six hours—and had good results on all of them. Great at design iteration Opus 4.5 is incredibly skilled at iterating through a design autonomously using an MCP like Playwright. previous models would lose the thread after a few cycles, or say a design was done when it wasn't. Opus 4.5 is incredible at autonomously iterating until a design is pixel perfect. we have a full 4,000 word vibe check on Every 📧 right now with everything we tested:

Dan Shipper 📧

272,699 views • 8 months ago

The verdict is in! Frontier models can pass the bar, yet they struggle on comprehensive legal research Today we're releasing Legal Research Bench, a benchmark that measures models’ ability to solve realistic legal research tasks across eight areas of U.S. law Instead of awarding partial credit, Legal Research Bench measures whether a model can conduct exhaustive legal analysis. We grade against a strict, all-pass rubric written by practicing lawyers. A model only receives full credit if every required legal element is correct Claude Opus 4.8 leads with 43.8% all-pass accuracy, followed by GPT 5.5 (40.4%) and Claude Sonnet 4.6 (38.5%). While top models score around 80% with partial credit, none exceed 44% when every required legal element must be correct The gap between partial and all-pass accuracy shows how difficult it remains for AI to produce complete, reliable legal research. We hope that Legal Research Bench helps better measure, and ultimately close that gap Lots of exciting work happening in Legal AI from Harvey and Crosby. Excited for the legal research benchmarks ahead!

The verdict is in! Frontier models can pass the bar, yet they struggle on comprehensive legal research Today we're releasing Legal Research Bench, a benchmark that measures models’ ability to solve realistic legal research tasks across eight areas of U.S. law Instead of awarding partial credit, Legal Research Bench measures whether a model can conduct exhaustive legal analysis. We grade against a strict, all-pass rubric written by practicing lawyers. A model only receives full credit if every required legal element is correct Claude Opus 4.8 leads with 43.8% all-pass accuracy, followed by GPT 5.5 (40.4%) and Claude Sonnet 4.6 (38.5%). While top models score around 80% with partial credit, none exceed 44% when every required legal element must be correct The gap between partial and all-pass accuracy shows how difficult it remains for AI to produce complete, reliable legal research. We hope that Legal Research Bench helps better measure, and ultimately close that gap Lots of exciting work happening in Legal AI from Harvey and Crosby. Excited for the legal research benchmarks ahead!

Vals AI

17,351 views • 1 month ago

Epoch AI researcher, Greg Burnham on the board game his team is using to prove AI still can't learn from experience like humans do: "We used the board game Earthborne Rangers as a test case for, if we give AI something unfamiliar, can it get better than its starting point?" "We chose a board game that can be played in solo mode, has relatively little spatial or visual reasoning. This game is based on building a deck and playing through cards, and you're exploring a wilderness landscape. Our benchmark covers five days of this game." "Out of the box, AI comes out pretty strong. It's probably playing better than a human would on a human's first try. But then we say, okay, keep playing. Take notes as you go. You're going to try to get better." "We basically just see flat performance. They don't make friends with the board game and figure out how it works and get better at it over time." "There's 21 objectives. Our best human run ever hits 20 out of 21. The AI systems can get 10, 11, 12 on an early run, but we let them play like 30 times, and they don't get any better than that." "This is a result more generally about learning on the fly. People call this continual learning. It's a very strong capability. If AI ever develops it, it kind of really upends a lot of assumptions we have. But so far this seems to not be a capability that we're observing." Greg Burnham Epoch AI

Epoch AI researcher, Greg Burnham on the board game his team is using to prove AI still can't learn from experience like humans do: "We used the board game Earthborne Rangers as a test case for, if we give AI something unfamiliar, can it get better than its starting point?" "We chose a board game that can be played in solo mode, has relatively little spatial or visual reasoning. This game is based on building a deck and playing through cards, and you're exploring a wilderness landscape. Our benchmark covers five days of this game." "Out of the box, AI comes out pretty strong. It's probably playing better than a human would on a human's first try. But then we say, okay, keep playing. Take notes as you go. You're going to try to get better." "We basically just see flat performance. They don't make friends with the board game and figure out how it works and get better at it over time." "There's 21 objectives. Our best human run ever hits 20 out of 21. The AI systems can get 10, 11, 12 on an early run, but we let them play like 30 times, and they don't get any better than that." "This is a result more generally about learning on the fly. People call this continual learning. It's a very strong capability. If AI ever develops it, it kind of really upends a lot of assumptions we have. But so far this seems to not be a capability that we're observing." Greg Burnham Epoch AI

MTS

13,050 views • 23 days ago

This is one of the craziest AI launches of 2026 and it came out of basically nowhere (Save this). A company called Subquadratic just shipped SubQ, and the benchmarks are almost hard to believe. To understand why this is such a big deal, you have to understand the fundamental problem that has defined AI for the last decade. Every large language model in existence is built on transformer architecture, and transformers use a mechanism called standard attention that checks every single word in a sequence against every other word. Double the context length and compute doesn't double, it quadruples, triple it and compute goes up nine times. This quadratic scaling is why frontier models have been stuck at roughly 1 million tokens, why running them at those lengths gets expensive fast, and why the AI labs have essentially been printing money charging you more the longer you need the model to think. The industry has known this problem existed since 2017 but they scaled it anyway. SubQ is built from the ground up to solve it. Instead of processing every possible token relationship, SubQ's sparse attention architecture identifies which relationships actually matter and ignores the rest meaning compute is used where it counts and wasted nowhere else. The result is that compute scales linearly with context length instead of exponentially, and the implications of that one architectural shift are enormous. At 12 million tokens, SubQ reduces attention compute by nearly 1,000x compared to standard frontier models and at 1 million tokens, it runs 52x faster than FlashAttention. And it does all of this while posting frontier level accuracy, scoring 95% on the RULER 128K long-context benchmark versus Claude Opus 4.6's 94.8%, and an 81.8 on SWE-Bench Verified coding tasks, besting Opus 4.6 (80.8) and DeepSeek 4.0 Pro. The cost comparison is where it gets genuinely insane. SubQ runs at under $1.50 per million tokens less than 5% of what Claude Opus charges. On the RULER benchmark, running the test with SubQ cost $8, running the same test with Claude Opus cost $2,600 and that's a 300x cost reduction at equivalent or better accuracy.. Subquadratic launched with $29 million in funding, SubQ is available today for early access via API, and SubQ Code, a coding agent built on the architecture ships alongside it. The transformer has been the unchallenged foundation of every major AI system since 2017. SubQ is the first serious evidence that something structurally better might have just arrived.

This is one of the craziest AI launches of 2026 and it came out of basically nowhere (Save this). A company called Subquadratic just shipped SubQ, and the benchmarks are almost hard to believe. To understand why this is such a big deal, you have to understand the fundamental problem that has defined AI for the last decade. Every large language model in existence is built on transformer architecture, and transformers use a mechanism called standard attention that checks every single word in a sequence against every other word. Double the context length and compute doesn't double, it quadruples, triple it and compute goes up nine times. This quadratic scaling is why frontier models have been stuck at roughly 1 million tokens, why running them at those lengths gets expensive fast, and why the AI labs have essentially been printing money charging you more the longer you need the model to think. The industry has known this problem existed since 2017 but they scaled it anyway. SubQ is built from the ground up to solve it. Instead of processing every possible token relationship, SubQ's sparse attention architecture identifies which relationships actually matter and ignores the rest meaning compute is used where it counts and wasted nowhere else. The result is that compute scales linearly with context length instead of exponentially, and the implications of that one architectural shift are enormous. At 12 million tokens, SubQ reduces attention compute by nearly 1,000x compared to standard frontier models and at 1 million tokens, it runs 52x faster than FlashAttention. And it does all of this while posting frontier level accuracy, scoring 95% on the RULER 128K long-context benchmark versus Claude Opus 4.6's 94.8%, and an 81.8 on SWE-Bench Verified coding tasks, besting Opus 4.6 (80.8) and DeepSeek 4.0 Pro. The cost comparison is where it gets genuinely insane. SubQ runs at under $1.50 per million tokens less than 5% of what Claude Opus charges. On the RULER benchmark, running the test with SubQ cost $8, running the same test with Claude Opus cost $2,600 and that's a 300x cost reduction at equivalent or better accuracy.. Subquadratic launched with $29 million in funding, SubQ is available today for early access via API, and SubQ Code, a coding agent built on the architecture ships alongside it. The transformer has been the unchallenged foundation of every major AI system since 2017. SubQ is the first serious evidence that something structurally better might have just arrived.

Milk Road AI

278,367 views • 3 months ago