Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Someone built a plugin that cuts Claude Code's cost by 54% on the same task, same model. Not a different model. Not a different harness. Just smarter tools underneath. It's called WOZCODE and the benchmark numbers are genuinely CRAZYYYY.

Hasan Toor

437,709 subscribers

20,754 views • 5 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

This completely changes how I look at AI benchmarks. Qwen 3.8 Max Preview felt nearly unusable in OpenCode and Qwen Code. With the Claude Code harness, it suddenly came surprisingly close to Kimi K3 and Fable 5. Same model. Same task. Different harness. Wildly different result. How is this even possible?

This completely changes how I look at AI benchmarks. Qwen 3.8 Max Preview felt nearly unusable in OpenCode and Qwen Code. With the Claude Code harness, it suddenly came surprisingly close to Kimi K3 and Fable 5. Same model. Same task. Different harness. Wildly different result. How is this even possible?

OmedTheVibeCoder

30,223 views • 2 months ago

Your AI sucks. And it's not the model. It's you. "Make it better" isn't a prompt, it's a guessing game. I changed one step in my process and the output quality completely flipped. Same ideas, same models, totally different results.

Your AI sucks. And it's not the model. It's you. "Make it better" isn't a prompt, it's a guessing game. I changed one step in my process and the output quality completely flipped. Same ideas, same models, totally different results.

Flick

322,001 views • 2 months ago

You can smell a big model. Not the parameter count. Not the benchmark score. It's that feeling when something is actually reasoning. Not just pattern matching. We call it "big model smell."

You can smell a big model. Not the parameter count. Not the benchmark score. It's that feeling when something is actually reasoning. Not just pattern matching. We call it "big model smell."

Arena.ai

112,227 views • 5 months ago

Sub-agent Model Selection — Different Tasks, Different Models Your main agent runs Qwen3.6-Plus for quality. But not every subtask needs a flagship model. Now sub-agents can use a different model. Create a skill file with model: openai:qwen3.5-plus and the sub-agent runs on that model. Powerful model for the hard parts, fast model for the easy parts. Save tokens without sacrificing quality on what matters.

Sub-agent Model Selection — Different Tasks, Different Models Your main agent runs Qwen3.6-Plus for quality. But not every subtask needs a flagship model. Now sub-agents can use a different model. Create a skill file with model: openai:qwen3.5-plus and the sub-agent runs on that model. Powerful model for the hard parts, fast model for the easy parts. Save tokens without sacrificing quality on what matters.

Qwen

21,333 views • 5 months ago

Hit your usage limit in Codex/Claude Code mid-task? Orca now lets you resume the exact same session in a different agent harness — one click. Codex ⇄ Claude Code, full context intact🔥

Hit your usage limit in Codex/Claude Code mid-task? Orca now lets you resume the exact same session in a different agent harness — one click. Codex ⇄ Claude Code, full context intact🔥

Orca ADE

54,643 views • 2 months ago

Opus 4.7 - 400k vs 1m context - is there a difference? I've heard Theo - t3.gg talk about the fact that it is unlikely that Anthropic would have offered up a model with 1m context at the same cost, if it wasn't a different (i.e. cheaper to serve) model. I did a test where I toggled the 1m default model on & off in Claude Code (otherwise default settings, xHigh reasoning) and compared the outputs with 3x generations - same prompts etc. My observations: - Models feel DIFFERENT - often when you ask a model for the same generation, you get a somewhat different answer, but it feels & smells the same. Here 400k and 1m are very different every time - 400k model seems better - not that 1m is trash and 400k is amazing, but there are definitely issues with the level of ambition and accuracy that 1m model seems to have Examples of 1m failing: - Voxel Rome: the colosseum is nowhere near as impressive - Golden Gate: cars go sideways, waves not very high, bridge goes into land; though the structure of the bridge is a bit better - Stonehenge: structure is more 'wrong', lighting, shadows & textures are more flat and not as rich This isn't a conclusive evidence of course, but at least to me the two models do not behave the same way. Anecdotally as well when building 1m felt like it was doing more weird validation (e.g. going around in circles) and 400k was more straightforward. These sorts of things are harder to capture in tests, but you'd notice in Claude Code. You can review the hosted generations, see the code & prompts in the links below

Opus 4.7 - 400k vs 1m context - is there a difference? I've heard Theo - t3.gg talk about the fact that it is unlikely that Anthropic would have offered up a model with 1m context at the same cost, if it wasn't a different (i.e. cheaper to serve) model. I did a test where I toggled the 1m default model on & off in Claude Code (otherwise default settings, xHigh reasoning) and compared the outputs with 3x generations - same prompts etc. My observations: - Models feel DIFFERENT - often when you ask a model for the same generation, you get a somewhat different answer, but it feels & smells the same. Here 400k and 1m are very different every time - 400k model seems better - not that 1m is trash and 400k is amazing, but there are definitely issues with the level of ambition and accuracy that 1m model seems to have Examples of 1m failing: - Voxel Rome: the colosseum is nowhere near as impressive - Golden Gate: cars go sideways, waves not very high, bridge goes into land; though the structure of the bridge is a bit better - Stonehenge: structure is more 'wrong', lighting, shadows & textures are more flat and not as rich This isn't a conclusive evidence of course, but at least to me the two models do not behave the same way. Anecdotally as well when building 1m felt like it was doing more weird validation (e.g. going around in circles) and 400k was more straightforward. These sorts of things are harder to capture in tests, but you'd notice in Claude Code. You can review the hosted generations, see the code & prompts in the links below

Peter Gostev (SF: 22-26 June)

29,203 views • 4 months ago

MOST PEOPLE BUILD AI AGENTS BACKWARDS. They start with prompts, then add tools, memory, subagents, dashboards, and a graph nobody can debug. Then they wonder why the agent is expensive, confused, and unreliable. The better order is simple: harness first loop second graph only when needed A harness gives the model a workspace, tools, permissions, logs, and a definition of done. A loop lets it try, fail, read feedback, repair, and stop. A graph only makes sense when the workflow branches: approvals, parallel work, durable state, different routes, restart recovery. A graph is not a smarter prompt. A loop is not a production system. A harness is not just a system prompt. Most agents do not fail because the model is weak. They fail because nobody defined what success means.

MOST PEOPLE BUILD AI AGENTS BACKWARDS. They start with prompts, then add tools, memory, subagents, dashboards, and a graph nobody can debug. Then they wonder why the agent is expensive, confused, and unreliable. The better order is simple: harness first loop second graph only when needed A harness gives the model a workspace, tools, permissions, logs, and a definition of done. A loop lets it try, fail, read feedback, repair, and stop. A graph only makes sense when the workflow branches: approvals, parallel work, durable state, different routes, restart recovery. A graph is not a smarter prompt. A loop is not a production system. A harness is not just a system prompt. Most agents do not fail because the model is weak. They fail because nobody defined what success means.

Fluixo

11,315 views • 1 month ago

I made Claude compete against itself. The smartest model does not automatically make the best agent. To prove it, I took the same Opus 4.6 model, initial prompt, and empty repo and but it inside two different coding harnesses: Claude Code vs. Factory Droid. The task: clone Excalidraw from scratch, inspect the original in a browser, implement its key interactions, and verify the result. Factory Droid: 8 minutes, 20 tool calls, $1.60 Claude Code: 19 minutes, 40 tool calls, $1.87 The only difference was the harness. Get Free Factory credits here:

I made Claude compete against itself. The smartest model does not automatically make the best agent. To prove it, I took the same Opus 4.6 model, initial prompt, and empty repo and but it inside two different coding harnesses: Claude Code vs. Factory Droid. The task: clone Excalidraw from scratch, inspect the original in a browser, implement its key interactions, and verify the result. Factory Droid: 8 minutes, 20 tool calls, $1.60 Claude Code: 19 minutes, 40 tool calls, $1.87 The only difference was the harness. Get Free Factory credits here:

Sarah Chieng

92,118 views • 28 days ago

wow... I just cut my claude code per-call cost to almost nothing with a 30-second config swap. Step 3.5 from StepFun plugs into your existing ~/.claude/settings.json by swapping three env vars. same workflow, different backend underneath. my agents have not stalled once since the switch. full setup below 🧵

wow... I just cut my claude code per-call cost to almost nothing with a 30-second config swap. Step 3.5 from StepFun plugs into your existing ~/.claude/settings.json by swapping three env vars. same workflow, different backend underneath. my agents have not stalled once since the switch. full setup below 🧵

Farhan

17,871 views • 3 months ago

I ran the same task on Claude Code and DeepSeek's new agent harness. One cost $150. The other cost $2. Today we're launching (AgentSky), the "OpenRouter for Agents" — one API → Claude Code, Codex, DeepSeek, Kimi, OpenCode, and every major agent in the cloud. And Agent Playground on top: race them on your own task, with your real tools (GitHub, Gmail, more), side by side in a browser: time, cost, tokens burnt. Guess which one was $2.

I ran the same task on Claude Code and DeepSeek's new agent harness. One cost $150. The other cost $2. Today we're launching (AgentSky), the "OpenRouter for Agents" — one API → Claude Code, Codex, DeepSeek, Kimi, OpenCode, and every major agent in the cloud. And Agent Playground on top: race them on your own task, with your real tools (GitHub, Gmail, more), side by side in a browser: time, cost, tokens burnt. Guess which one was $2.

Xiaoyin Qu

2,222,331 views • 28 days ago

Anthropic pays its ML engineers $500K+/year to deploy what Percy Liang just taught for free in Lecture 3 of CS221. 80 minutes. one whiteboard. everything between "a neuron" and the model running in your IDE right now. three things click after this one: / why two Claude prompts that "mean the same thing" land in different regions of weight space / why fine-tuning sometimes makes a model dumber, not smarter / what RLHF actually changes when Anthropic aligns Claude same $90K Stanford sequence. lecture 3 of ~20. free. the gap between "using Claude" and "understanding it" is exactly one lecture wide.

Anthropic pays its ML engineers $500K+/year to deploy what Percy Liang just taught for free in Lecture 3 of CS221. 80 minutes. one whiteboard. everything between "a neuron" and the model running in your IDE right now. three things click after this one: / why two Claude prompts that "mean the same thing" land in different regions of weight space / why fine-tuning sometimes makes a model dumber, not smarter / what RLHF actually changes when Anthropic aligns Claude same $90K Stanford sequence. lecture 3 of ~20. free. the gap between "using Claude" and "understanding it" is exactly one lecture wide.

dunik

24,391 views • 4 months ago

AI isn't built by companies alone. It's built on the actions of millions of people. The problem? Millions contribute. A handful benefit. We're building a different future with Action Model.

AI isn't built by companies alone. It's built on the actions of millions of people. The problem? Millions contribute. A handful benefit. We're building a different future with Action Model.

Action Model

17,790 views • 1 month ago

alright agent nerds, if you care about your tokens and usage limits, pay attention to the tools you give to your agents. i built a benchmark that compared various browser tools for agents, and here's an example of their massive difference in cost and latency doing the same task

alright agent nerds, if you care about your tokens and usage limits, pay attention to the tools you give to your agents. i built a benchmark that compared various browser tools for agents, and here's an example of their massive difference in cost and latency doing the same task

Kun Chen

571,248 views • 5 months ago

Same model psd for both Same rigger Left is 2022 and the one on the Right is 2025 a few days ago It's crazy how much a riggers can really change how a model looks LIKE THAT DOESN'T EVEN LOOK LIKE THE SAME MODEL, BUT IT IS!!

Same model psd for both Same rigger Left is 2022 and the one on the Right is 2025 a few days ago It's crazy how much a riggers can really change how a model looks LIKE THAT DOESN'T EVEN LOOK LIKE THE SAME MODEL, BUT IT IS!!

💙💉 Khione || Vtuber

86,688 views • 1 year ago

watch this anon. i gave NVIDIA's biggest model ever a single task. 100 minutes and 440,000 tokens later, it had rendered nothing. not one important thing on the screen. this is Nemotron 3 Ultra. 550 billion parameters, a hybrid Mamba Transformer MoE, the largest model NVIDIA has ever shipped, and they built it specifically for long-running agentic coding. so i handed it exactly that: build a 3D scene from a spec, multiple files, iterate until the tests pass. the same task a frontier model one shotted in minutes. i genuinely wanted to be impressed. it ran for an hour and forty. burned through 440,000 tokens. wrote every file, passed its own tests, and proudly printed "task complete."the browser was blank. the 3D scene never rendered. not once. and the long horizon agentic behavior was genuinely good. it stayed on task the whole hour and forty, wrote real multi-file code, drove its own tools without derailing. it just couldn't turn any of that into something that actually runs. here's the part that gets me. it's a text model, it cannot see its own output. so it sat there looping on a broken vision tool, trying to "look" at the page, hitting error after error, never once reasoning its way out. it declared victory on an empty screen because it had no way to know the screen was empty. to be fair, i genuinely don't know what quant the NIM was serving, so maybe some of that's on the serving, not the model. but the biggest model NVIDIA has ever made, on the exact task it was designed for, couldn't tell it had built nothing in 100 minutes. same task on a local model, below thread👇.

watch this anon. i gave NVIDIA's biggest model ever a single task. 100 minutes and 440,000 tokens later, it had rendered nothing. not one important thing on the screen. this is Nemotron 3 Ultra. 550 billion parameters, a hybrid Mamba Transformer MoE, the largest model NVIDIA has ever shipped, and they built it specifically for long-running agentic coding. so i handed it exactly that: build a 3D scene from a spec, multiple files, iterate until the tests pass. the same task a frontier model one shotted in minutes. i genuinely wanted to be impressed. it ran for an hour and forty. burned through 440,000 tokens. wrote every file, passed its own tests, and proudly printed "task complete."the browser was blank. the 3D scene never rendered. not once. and the long horizon agentic behavior was genuinely good. it stayed on task the whole hour and forty, wrote real multi-file code, drove its own tools without derailing. it just couldn't turn any of that into something that actually runs. here's the part that gets me. it's a text model, it cannot see its own output. so it sat there looping on a broken vision tool, trying to "look" at the page, hitting error after error, never once reasoning its way out. it declared victory on an empty screen because it had no way to know the screen was empty. to be fair, i genuinely don't know what quant the NIM was serving, so maybe some of that's on the serving, not the model. but the biggest model NVIDIA has ever made, on the exact task it was designed for, couldn't tell it had built nothing in 100 minutes. same task on a local model, below thread👇.

Sudo su

32,589 views • 2 months ago

“The model alone is no longer the product" Aravind Srinivas says the real product is now the harness around it: orchestration, tools, enterprise context, and cost performance. The post-frontier AI race is about systems, not just the smartest model

“The model alone is no longer the product" Aravind Srinivas says the real product is now the harness around it: orchestration, tools, enterprise context, and cost performance. The post-frontier AI race is about systems, not just the smartest model

Deirdre Bosa

102,508 views • 2 months ago

your agent reviewing its own work is not a check. it is a second opinion from the same source. this is the most common gap in agent systems and it hides in plain sight, because the step exists. there is a review. it just cannot do the thing you think it does. here is the mechanism. the model produced an output from a context. you then ask the same model, holding the same context, whether that output is correct. it answers fluently, because that is what it does. and the answer is drawn from the same distribution that produced the thing being judged. same weights, same window, same blind spots. if the reason the output is wrong is something the model does not know, the review does not know it either. if the reason is something the context does not contain, the review has the same context. the failure mode and the detector share a cause. > why it feels like it works because most of the time the output is fine, and the review says fine. agreement is not evidence of detection. a reviewer that says pass on everything agrees with reality most of the time too. what you actually want to measure is what happens on the cases that are wrong. that is the only place a check earns its name, and it is exactly the place where a self-review is weakest. there is research on this. Huang and colleagues at DeepMind showed at ICLR 2024 that intrinsic self-correction, revising without external grounding, does not reliably help and often makes things worse. > what to actually do move the check outside the model. a test that runs, a schema that validates, a file that exists or does not, an exit code from something you did not write. these are not smarter than the model. they are just not correlated with it, and that is the entire value. when the judgement genuinely needs a model, at minimum use a different family. same family means shared blind spots, and frontier judges measurably inflate scores for outputs that look like their own. and split the work by kind. anything objectively checkable goes to code. only the genuinely semantic calls go to a judge, and those get a rubric written as one line. a review inside the loop tells you the model is confident. a check outside it tells you whether the work is done. save this - then read the eval setup below

your agent reviewing its own work is not a check. it is a second opinion from the same source. this is the most common gap in agent systems and it hides in plain sight, because the step exists. there is a review. it just cannot do the thing you think it does. here is the mechanism. the model produced an output from a context. you then ask the same model, holding the same context, whether that output is correct. it answers fluently, because that is what it does. and the answer is drawn from the same distribution that produced the thing being judged. same weights, same window, same blind spots. if the reason the output is wrong is something the model does not know, the review does not know it either. if the reason is something the context does not contain, the review has the same context. the failure mode and the detector share a cause. > why it feels like it works because most of the time the output is fine, and the review says fine. agreement is not evidence of detection. a reviewer that says pass on everything agrees with reality most of the time too. what you actually want to measure is what happens on the cases that are wrong. that is the only place a check earns its name, and it is exactly the place where a self-review is weakest. there is research on this. Huang and colleagues at DeepMind showed at ICLR 2024 that intrinsic self-correction, revising without external grounding, does not reliably help and often makes things worse. > what to actually do move the check outside the model. a test that runs, a schema that validates, a file that exists or does not, an exit code from something you did not write. these are not smarter than the model. they are just not correlated with it, and that is the entire value. when the judgement genuinely needs a model, at minimum use a different family. same family means shared blind spots, and frontier judges measurably inflate scores for outputs that look like their own. and split the work by kind. anything objectively checkable goes to code. only the genuinely semantic calls go to a judge, and those get a rubric written as one line. a review inside the loop tells you the model is confident. a check outside it tells you whether the work is done. save this - then read the eval setup below

Hanako

14,325 views • 1 month ago

GPT-6 Astra built a full 3D model of the human cell in 30 minutes. Biology didn't see this coming. Not a diagram. Not an illustration. A complete, detailed model built from scratch in a single session. The same tool that handles code, design, and 3D geometry is now walking into science.

GPT-6 Astra built a full 3D model of the human cell in 30 minutes. Biology didn't see this coming. Not a diagram. Not an illustration. A complete, detailed model built from scratch in a single session. The same tool that handles code, design, and 3D geometry is now walking into science.

Zentrix⌚️

52,097 views • 13 days ago

It's the same policy as Trudeau, just with a different face.

It's the same policy as Trudeau, just with a different face.

Canada Proud

12,364 views • 1 year ago