正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

DeepSeek-V4.1-Flash: the BEST Boeing benchmark result I’ve seen from an open-source model (by far). ✈️ It ran for hours with /goal. Where other models often stall quickly, it kept improving: inspect, spot a defect, zoom in, diagnose, fix, repeat. The video really shows how strong it is at sustaining... show more

Victor M

30,659 subscribers

71,189 次观看 • 11 天前 •via X (Twitter)

游戏科学技术

Anya Rossi• Live Now

Private livecam show

33 条评论

Victor M 的头像

Victor M11 天前

more details because this is beautiful 🔥

Victor M 的头像

Victor M11 天前

extremely good, inspect the result here:

Offscript 的头像

Offscript11 天前

kept improving instead of stalling is the only benchmark that matters now. the rest is one shot theatre

Soft Axiom 的头像

Soft Axiom10 天前

"Until you are 100% satisfied" might be the most important part of the prompt. The model grading its own work is the benchmark now.

Myravyx 的头像

Myravyx10 天前

The hard part in a multi-hour run isn't the fixing. It's still being dissatisfied at hour three.

catman 的头像

catman11 天前

hours of /goal is a much harsher test than a one-shot render. what finally made it stop improving: a visual score, a fixed iteration limit, or its own judgment?

Umar Nasser 的头像

Umar Nasser10 天前

It seems better than opus 4.8 one right?

Shesaidmewakeup 的头像

Shesaidmewakeup10 天前

Open source running for hours in claude code and still improving. Benches don’t show that.

John Rood 的头像

John Rood11 天前

the thing that separates "keeps improving" from "stalls" isn't raw capability. it's whether the model can still see what it already tried 40 steps back. most stalls are it relitigating a fix it already ruled out because that context fell out of the window.

neal 的头像

neal11 天前

The impressive part is the loop, not the “god model” label: inspect → name a defect → fix → verify. That pattern is useful outside 3D work too—give any model explicit checks and a stop condition instead of asking for a one-shot answer.

OmniWired 的头像

OmniWired10 天前

I wonder if it be better on dsh

Surya Rajendhran 的头像

Surya Rajendhran11 天前

this is a great benchmark, where can I find more details? are there any common trends you’re seeing with how deepseek or open weight models handle 3D vs closed frontier models?

Victor M 的头像

Victor M11 天前

best way to see other results:

Chizzydigital 的头像

Chizzydigital10 天前

Next time just use a separate critic agent

veloX 的头像

veloX10 天前

this Boeing benchmark reminds me of the Red Queen Gödel Machine: continuous self-improvement against a moving target.

bbzz 的头像

bbzz11 天前

what'd it cost?

Elledyne Labs 的头像

Elledyne Labs10 天前

was the inspect→fix loop running on the same context window the whole time, or did /goal spawn fresh subagents? curious what context strategy kept it from rotting across hours

Krish Dasgupta 的头像

Krish Dasgupta10 天前

What really is marvellous- the precision shadow. The moment you move the pane around the plane, the direction also specifies the shadow. That’s something !

Wei佳的头像

Wei佳10 天前

What Harness tool are you using?

Elledyne Labs 的头像

Elledyne Labs10 天前

the harness did the work, not the model. claude code gave it the loop and deepseek just refused to quit. how much of this is /goal's verifier vs deepseek's stamina?

اسلیم کلیک ها 的头像

اسلیم کلیک ها11 天前

Can you please crash it into a building next

Onur Solmaz 的头像

Onur Solmaz10 天前

did it reach the final version faster than astra before?

Steffan 的头像

Steffan11 天前

Looks like I am going to change harness, I was not really happy with deepseek results in opencode, maybe it’s a harness problem.

Victor M 的头像

Victor M10 天前

try it and tell me - I get really good results with CC with this setup:

FoodTruck Bench 的头像

FoodTruck Bench10 天前

DeepSeek 4.1 being this close to GPT-5.5 at this price point is actually illegal 💀

josip 的头像

josip10 天前

the impressive part isn't the boeing render, it's that it stayed in the inspect→diagnose→fix loop for hours instead of stalling after the first pretty output. long-horizon models start looking useful when the harness keeps them pointed at the next defect

Dev 的头像

Dev10 天前

bro chinese models r literally cooking the entire ai market rn distillation really leveled up chinese ai to the point it's outperforming everything ngl 😭🔥

kimodo_cloud 的头像

kimodo_cloud11 天前

cost ?

Preyforge 的头像

Preyforge10 天前

this is the satisfying version of a loop: inspect, diagnose, fix. i still make the deployed path prove the demo; screenshots can be charming little liars.

neo hodler 的头像

neo hodler10 天前

Interesting to see that it “ran for hours with /goal” without stalling. Long-horizon loops are where most “SOTA” models still quietly die.

Ibesh 的头像

Ibesh10 天前

boeing test is brutal lol. most open models stall after a few fixes, the fact it kept going for hours and kept improving is the real flex

keethesh 的头像

keethesh10 天前

Does it ever break something that was already working?

东倒西六条的头像

东倒西六条10 天前

你炒股吗？如果你炒股的话马上删了，否则对今晚美国股市有冲击！能设计外形，就可以对飞机内部进行设计。直接冲击了美国高端制造业

相关视频

Old way of using Claude Code: Prompt → wait → review → prompt → wait → review → repeat 20 times. You are the loop. You are the condition check. You are the reason it keeps going. New way: /goal all tests pass and lint is clean Claude works. You do something else. You come back to a result. Here's exactly how to set it up — and how to write conditions that actually work ↓

Old way of using Claude Code: Prompt → wait → review → prompt → wait → review → repeat 20 times. You are the loop. You are the condition check. You are the reason it keeps going. New way: /goal all tests pass and lint is clean Claude works. You do something else. You come back to a result. Here's exactly how to set it up — and how to write conditions that actually work ↓

Rahul

38,382 次观看 • 4 个月前

Cat Wu, head of product for Claude Code at Anthropic: "you can build almost anything with Claude from a single prompt now" one prompt gets you a working screen, not YOUR product Claude Code has never seen your design system, so it guesses the look fresh every time. how to hand Claude Code that system is in the article. better than the $300 design course in your tabs, and this one's free.

Cat Wu, head of product for Claude Code at Anthropic: "you can build almost anything with Claude from a single prompt now" one prompt gets you a working screen, not YOUR product Claude Code has never seen your design system, so it guesses the look fresh every time. how to hand Claude Code that system is in the article. better than the $300 design course in your tabs, and this one's free.

Mnimiy

114,141 次观看 • 3 个月前

Jenny Wen, head of design for Claude at Anthropic: "the design process is dead." what replaces it runs on machine-readable design systems In a 1-hour conversation, Jenny Wen breaks down why the discovery -> mock -> iterate loop is dying and what designers do instead. design system -> Claude Code -> every screen fixed at once, not one at a time i rebuilt the same 10-screen dashboard twice: ~6 hours of restyling by hand vs ~40 minutes when the fix lived in the system. how to build that system and hand it to Claude Code is in the article below.

Jenny Wen, head of design for Claude at Anthropic: "the design process is dead." what replaces it runs on machine-readable design systems In a 1-hour conversation, Jenny Wen breaks down why the discovery -> mock -> iterate loop is dying and what designers do instead. design system -> Claude Code -> every screen fixed at once, not one at a time i rebuilt the same 10-screen dashboard twice: ~6 hours of restyling by hand vs ~40 minutes when the fix lived in the system. how to build that system and hand it to Claude Code is in the article below.

Mnimiy

180,843 次观看 • 3 个月前

What is a software factory? It is your workflow, your skills, and your domain knowledge, saved in files that the agent reads so the whole process is productized Think about the last app you built. You typed what you wanted, and the agent built it. You did not like it, so you made changes. You shipped it to Vercel, or you kept going. A factory puts structure on each of those steps. At each step you ask how to get the most from the model, and how to move faster. That is the conveyor belt. You ship quickly, and the quality stays the same. Mine is 5 md files. It works in Codex, Claude Code, and Cursor, because it does not depend on the model or the tool. Build your own.

What is a software factory? It is your workflow, your skills, and your domain knowledge, saved in files that the agent reads so the whole process is productized Think about the last app you built. You typed what you wanted, and the agent built it. You did not like it, so you made changes. You shipped it to Vercel, or you kept going. A factory puts structure on each of those steps. At each step you ask how to get the most from the model, and how to move faster. That is the conveyor belt. You ship quickly, and the quality stays the same. Mine is 5 md files. It works in Codex, Claude Code, and Cursor, because it does not depend on the model or the tool. Build your own.

The Startup Ideas Podcast (SIP) 🧃

17,107 次观看 • 7 天前

She is only 17 built her first system with Opus 5 sold it to Anthropic for $1.2M - and at Stanford shared how: 00:20 - how Opus 5 builds a $1.2M system in 60 minutes 23:34 - how the system replaced 50 Anthropic engineers and saves millions 52:47 - the Claude Code setting that turns a prompt into a $100k+ system after watching I gave Opus 5 an idea I never knew how to build - by evening it was created and sent to Anthropic for sale. save & watch - the article below shows how Claude Code builds a system in one evening that Anthropic will buy for $1.2M.

She is only 17 built her first system with Opus 5 sold it to Anthropic for $1.2M - and at Stanford shared how: 00:20 - how Opus 5 builds a $1.2M system in 60 minutes 23:34 - how the system replaced 50 Anthropic engineers and saves millions 52:47 - the Claude Code setting that turns a prompt into a $100k+ system after watching I gave Opus 5 an idea I never knew how to build - by evening it was created and sent to Anthropic for sale. save & watch - the article below shows how Claude Code builds a system in one evening that Anthropic will buy for $1.2M.

Noisy

40,915 次观看 • 18 天前

THE GUY RUNNING CLAUDE CODE AT ANTHROPIC JUST SAID HE DOESN’T PROMPT AI ANYMORE. He writes loops. That’s a huge difference. A prompt tells Claude what to do once. A loop gives Claude a goal. Then Claude keeps checking its own work… Finding mistakes. Writing its next prompt. Trying again. Over and over… Until it actually reaches the target you defined. But here’s what most people miss. A loop is only as smart as the context you give it. That’s exactly why Claude Repositories matter. I broke down how to connect your entire codebase so Claude can actually reason across your project instead of a single file. Full guide below

THE GUY RUNNING CLAUDE CODE AT ANTHROPIC JUST SAID HE DOESN’T PROMPT AI ANYMORE. He writes loops. That’s a huge difference. A prompt tells Claude what to do once. A loop gives Claude a goal. Then Claude keeps checking its own work… Finding mistakes. Writing its next prompt. Trying again. Over and over… Until it actually reaches the target you defined. But here’s what most people miss. A loop is only as smart as the context you give it. That’s exactly why Claude Repositories matter. I broke down how to connect your entire codebase so Claude can actually reason across your project instead of a single file. Full guide below

kiosa

16,808 次观看 • 2 个月前

AI AGENTS 101 (58 minute free masterclass) send this to anyone who wants to understand ai agents, claude skills, md files, how to get the most out of AI etc in plain english: 1. chat vs agents - chat models answer questions in a back and forth while agents take a goal, figure out the steps, and deliver a result 2. agents don’t stop after one response. they keep running until the task is actually finishedno babysitting required 3. everything runs on a loop. they gather context, decide what to do, take an action, then repeat until done 4. the loop is the system. they look at files, tools, and the internet. decide the next step. execute and then feed that back into the next step. over and over until completion 5. the model is just one piece. gpt, claude, gemini are the reasoning layer. the key is model + loop + tools + context 6. mcp is how agents use tools. it connects things like browser, code, apis, and your internal software. once connected, the agent decides when to use them to get the job done 7. context beats prompt all day. you don't need to write perfect prompts. load your agent with context about your business, style, and goals and then simple instructions work 8. claude.md or agents.md is the onboarding doc it tells the agent who it is, how to behave, what it knows, and what tools it can use. this gets loaded every time before it starts 9. memory.md is how it improves. agents don’t remember by default. this file stores preferences, corrections, and patterns you tell the agent to update it, and it gets better over time 10. skills + harnesses make it usable. skills are reusable tasks like writing, research, analysis the harness is the environment like claude code or openclaw that runs everything. basiclaly, different interfaces, same system underneath this episode with remy on The Startup Ideas Podcast (SIP) 🧃 was one of the clearest ways of understanding a lot of the core concepts of ai agents could be the best beginners course for ai agents 58 mins. all free. no advertisers. i just want to see you build cool stuff. im rooting for you. send to a friend watch

AI AGENTS 101 (58 minute free masterclass) send this to anyone who wants to understand ai agents, claude skills, md files, how to get the most out of AI etc in plain english: 1. chat vs agents - chat models answer questions in a back and forth while agents take a goal, figure out the steps, and deliver a result 2. agents don’t stop after one response. they keep running until the task is actually finishedno babysitting required 3. everything runs on a loop. they gather context, decide what to do, take an action, then repeat until done 4. the loop is the system. they look at files, tools, and the internet. decide the next step. execute and then feed that back into the next step. over and over until completion 5. the model is just one piece. gpt, claude, gemini are the reasoning layer. the key is model + loop + tools + context 6. mcp is how agents use tools. it connects things like browser, code, apis, and your internal software. once connected, the agent decides when to use them to get the job done 7. context beats prompt all day. you don't need to write perfect prompts. load your agent with context about your business, style, and goals and then simple instructions work 8. claude.md or agents.md is the onboarding doc it tells the agent who it is, how to behave, what it knows, and what tools it can use. this gets loaded every time before it starts 9. memory.md is how it improves. agents don’t remember by default. this file stores preferences, corrections, and patterns you tell the agent to update it, and it gets better over time 10. skills + harnesses make it usable. skills are reusable tasks like writing, research, analysis the harness is the environment like claude code or openclaw that runs everything. basiclaly, different interfaces, same system underneath this episode with remy on The Startup Ideas Podcast (SIP) 🧃 was one of the clearest ways of understanding a lot of the core concepts of ai agents could be the best beginners course for ai agents 58 mins. all free. no advertisers. i just want to see you build cool stuff. im rooting for you. send to a friend watch

GREG ISENBERG

377,138 次观看 • 6 个月前

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

MTS

61,234 次观看 • 2 个月前

You can now fine-tune Llama 3 without writing a single line of code! We are moving at breakneck speed. I recorded a video to show you how to fine-tune any open-source model in a few minutes. I'm using a GPT capable of taking a problem and turning it into a fine-tuned model that will solve it. You don't have to write any code. You only need to explain to a GPT what problem you want to solve and tell it you want to use Llama 3. For example, "fine-tune Llama 3" or "deploy zephyr." It feels magic. The system will recommend a dataset and fine-tune the model for you. I'm using Monster API, a platform that specializes in making fine-tuning and deploying open-source models easy and fast. Their stack is well-optimized to maximize fine-tuning efficiency using techniques like Q-Lora and vLLM. They are behind the GPT. Here is what you need to do: 1. Create an account at 2. Load the GPT with the link below This is as simple as it gets. When you are done, you can click a button to deploy the model and start using it. I have 10,000 free credits for anyone using the code "SANTIAGO" in the dashboard. You can use these credits to access, fine-tune, and deploy these open-source models. You can also keep up with their latest updates, and get free credits and special offers on their Discord server:

You can now fine-tune Llama 3 without writing a single line of code! We are moving at breakneck speed. I recorded a video to show you how to fine-tune any open-source model in a few minutes. I'm using a GPT capable of taking a problem and turning it into a fine-tuned model that will solve it. You don't have to write any code. You only need to explain to a GPT what problem you want to solve and tell it you want to use Llama 3. For example, "fine-tune Llama 3" or "deploy zephyr." It feels magic. The system will recommend a dataset and fine-tune the model for you. I'm using Monster API, a platform that specializes in making fine-tuning and deploying open-source models easy and fast. Their stack is well-optimized to maximize fine-tuning efficiency using techniques like Q-Lora and vLLM. They are behind the GPT. Here is what you need to do: 1. Create an account at 2. Load the GPT with the link below This is as simple as it gets. When you are done, you can click a button to deploy the model and start using it. I have 10,000 free credits for anyone using the code "SANTIAGO" in the dashboard. You can use these credits to access, fine-tune, and deploy these open-source models. You can also keep up with their latest updates, and get free credits and special offers on their Discord server:

Santiago

324,621 次观看 • 2 年前

CLAUDE CODE JUST SHIPPED THE FEATURE THAT SOLVES THE BIGGEST PROBLEM EVERY BUILDER HAS WITH AI AGENTS. The problem: Claude starts a task, gets distracted by a sub-problem, goes down a rabbit hole, and never finishes the original thing you asked for. The solution: /goal One command. You set the goal at the start of the session. Claude now has a north star it checks against every action it takes. Not just at the beginning. Throughout the entire session. Every time Claude is about to do something it asks: does this action move me toward the goal the user set or am I drifting? If it is drifting it corrects. If it completes a sub-task it returns to the primary goal. If it hits a blocker it reports back instead of spending 45 minutes solving the wrong problem. This sounds like a small feature. It is not. The reason most people do not trust Claude Code for long autonomous runs is not capability. It is reliability. A Claude Code session that reliably finishes what it started is worth 10 times more than one that is more capable but wanders. /goal is the feature that makes long autonomous sessions reliable. Set the goal. Let it run. Come back to a finished result. Not a result that got 70% done before Claude decided the sub-problem was more interesting. Done. The builders running overnight agent sessions are going to use this command on everything from today forward. Bookmark this. Follow CyrilXBT for every Claude Code feature the moment it ships.

CLAUDE CODE JUST SHIPPED THE FEATURE THAT SOLVES THE BIGGEST PROBLEM EVERY BUILDER HAS WITH AI AGENTS. The problem: Claude starts a task, gets distracted by a sub-problem, goes down a rabbit hole, and never finishes the original thing you asked for. The solution: /goal One command. You set the goal at the start of the session. Claude now has a north star it checks against every action it takes. Not just at the beginning. Throughout the entire session. Every time Claude is about to do something it asks: does this action move me toward the goal the user set or am I drifting? If it is drifting it corrects. If it completes a sub-task it returns to the primary goal. If it hits a blocker it reports back instead of spending 45 minutes solving the wrong problem. This sounds like a small feature. It is not. The reason most people do not trust Claude Code for long autonomous runs is not capability. It is reliability. A Claude Code session that reliably finishes what it started is worth 10 times more than one that is more capable but wanders. /goal is the feature that makes long autonomous sessions reliable. Set the goal. Let it run. Come back to a finished result. Not a result that got 70% done before Claude decided the sub-problem was more interesting. Done. The builders running overnight agent sessions are going to use this command on everything from today forward. Bookmark this. Follow CyrilXBT for every Claude Code feature the moment it ships.

CyrilXBT

19,586 次观看 • 4 个月前

Can't believe in less than 24 hours I created my own game with Claude and Three.js. Running in a browser at 120 FPS. All I had to do was give Claude the repo for Threejs and describe an MMO-like game for it to build from scratch. All the assets and mechanics you see are all from a prompt. As you can see, it is far from done but the progress is insane. Everyone should be using Three.js now without a doubt!

Can't believe in less than 24 hours I created my own game with Claude and Three.js. Running in a browser at 120 FPS. All I had to do was give Claude the repo for Threejs and describe an MMO-like game for it to build from scratch. All the assets and mechanics you see are all from a prompt. As you can see, it is far from done but the progress is insane. Everyone should be using Three.js now without a doubt!

NeilXbt

23,504 次观看 • 1 个月前

If you’ve never written code before, this is for you. I’ve just launched a course that shows you, in less than 30 minutes, how to describe an idea for an app and build it with AI. In this course, you'll build a working web application - a funny interactive birthday message generator that runs in your browser and can be shared with friends. You'll customize it by telling AI how you want it changed, and tweak it until it works the way you want. By the end, you'll have a repeatable process you can apply to build a wide variety of applications. If you want to try vibe coding, this will be the best place to start! Further, you'll be able to use these techniques with whatever tool you're most comfortable with (like ChatGPT, Gemini, Claude, or others) -- we're vendor neutral. Skills you'll gain: - How to build web apps with AI - zero coding skills needed - How to fix and improve your creations by chatting with AI - A simple process you can use to build other things you can dream up Building with AI is one of the most fun things in the world. Please join me and take your first step! I think you will be surprised at what you can build. And if you're an experienced engineer, please share this with someone in your life who's been curious about building with AI. Come build with me!

If you’ve never written code before, this is for you. I’ve just launched a course that shows you, in less than 30 minutes, how to describe an idea for an app and build it with AI. In this course, you'll build a working web application - a funny interactive birthday message generator that runs in your browser and can be shared with friends. You'll customize it by telling AI how you want it changed, and tweak it until it works the way you want. By the end, you'll have a repeatable process you can apply to build a wide variety of applications. If you want to try vibe coding, this will be the best place to start! Further, you'll be able to use these techniques with whatever tool you're most comfortable with (like ChatGPT, Gemini, Claude, or others) -- we're vendor neutral. Skills you'll gain: - How to build web apps with AI - zero coding skills needed - How to fix and improve your creations by chatting with AI - A simple process you can use to build other things you can dream up Building with AI is one of the most fun things in the world. Please join me and take your first step! I think you will be surprised at what you can build. And if you're an experienced engineer, please share this with someone in your life who's been curious about building with AI. Come build with me!

Andrew Ng

637,344 次观看 • 8 个月前

Claude Code creator: "100% of our pull requests at Anrtopic are run by Claude Code. 80–90% of code review too. The feature I’m using the most today is /loops. I’m not prompting Claude anymore - I’m building loops" in this interview he breaks down exactly how a system changes everything: - the memory and context features that turn Claude into a second brain - the knowledge architecture most users don't know Claude can build - the integration layer that connects Claude to your actual workflow - why typing one question at a time is the most expensive way to use Claude if you've been using Claude for months and still start every conversation from scratch with zero context, you don't have a Claude problem. you have a system problem instead of another show tonight, watch this make sure to bookmark it before it gets lost in your feed full guide in the article below

Claude Code creator: "100% of our pull requests at Anrtopic are run by Claude Code. 80–90% of code review too. The feature I’m using the most today is /loops. I’m not prompting Claude anymore - I’m building loops" in this interview he breaks down exactly how a system changes everything: - the memory and context features that turn Claude into a second brain - the knowledge architecture most users don't know Claude can build - the integration layer that connects Claude to your actual workflow - why typing one question at a time is the most expensive way to use Claude if you've been using Claude for months and still start every conversation from scratch with zero context, you don't have a Claude problem. you have a system problem instead of another show tonight, watch this make sure to bookmark it before it gets lost in your feed full guide in the article below

Khairallah AL-Awady

39,292 次观看 • 3 个月前

Before Fable got released (and pulled) Mozilla was quietly testing Claude Mythos against Firefox's 10M line codebase. The result? Over 400 security bugs fixes, including ones that had been hiding in the codebase for over a decade. Brian Grinstead, distinguished engineer at Mozilla, walked me through the agentic bug-finding harness behind the model. His take? It was 50% mythos / 50% setup. In this ep, Brian walks through: - why you can't just point a model at 10M lines of code - how to write a good goal/loop pattern - killing false positives with a verifier - why it's good to "lie to the agent" And guess what? This isn't magic - you can write your own similar harness in less than an afternoon. Watch now on YT:

Before Fable got released (and pulled) Mozilla was quietly testing Claude Mythos against Firefox's 10M line codebase. The result? Over 400 security bugs fixes, including ones that had been hiding in the codebase for over a decade. Brian Grinstead, distinguished engineer at Mozilla, walked me through the agentic bug-finding harness behind the model. His take? It was 50% mythos / 50% setup. In this ep, Brian walks through: - why you can't just point a model at 10M lines of code - how to write a good goal/loop pattern - killing false positives with a verifier - why it's good to "lie to the agent" And guess what? This isn't magic - you can write your own similar harness in less than an afternoon. Watch now on YT:

claire vo 🖤

260,235 次观看 • 3 个月前

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Akshay 🚀

76,998 次观看 • 1 个月前

72 hours in and GPT-5.6 Sol is still running without a single follow-up prompt. The task was to build the most comprehensive map possible of the physical infrastructure powering AI, submarine cables, data centers, power plants, substations and electricity grids. The only thing given was a simple loop: → Find the biggest gap → Research it → Build it → Test it → Publish it → Repeat Most people expected this to stall after a few hours. It didn't. The Mac restarted mid-run and Codex picked up exactly where it left off without losing anything. What this actually proves is that the prompt matters less than the goal. The smarter the model, the more room you need to give it to make its own decisions. Constrain it with steps and you get a worse version of what it could do on its own.

72 hours in and GPT-5.6 Sol is still running without a single follow-up prompt. The task was to build the most comprehensive map possible of the physical infrastructure powering AI, submarine cables, data centers, power plants, substations and electricity grids. The only thing given was a simple loop: → Find the biggest gap → Research it → Build it → Test it → Publish it → Repeat Most people expected this to stall after a few hours. It didn't. The Mac restarted mid-run and Codex picked up exactly where it left off without losing anything. What this actually proves is that the prompt matters less than the goal. The smarter the model, the more room you need to give it to make its own decisions. Constrain it with steps and you get a worse version of what it could do on its own.

0xMarioNawfal

61,751 次观看 • 2 个月前

deepseek v4.1 flash ran all four three.js scenes for $0.15. gemini 3.8 flash needed $1.25 we put the two models on one job: a self-contained html page that renders an animated 3d scene with three.js – a storm at sea, a spiral galaxy, a tornado over farmland, a tesla coil on a dark table. same prompt - calls until all four scenes were usable #1 deepseek v4.1 flash – 6 #2 gemini 3.8 flash – 12 - time #1 deepseek v4.1 flash – 13m 57s #2 gemini 3.8 flash – 32m 02s - cost #1 deepseek v4.1 flash – $0.15 #2 gemini 3.8 flash – $1.25 observations: • gemini blew out the exposure on 4 of 5 first attempts – a white blob where the tesla coil should be. the prompt had a paragraph on exposure. it did not help. • both models overcorrect in the fix loop: "too bright" becomes "almost black". round two is where it lands. same four scenes, 6 calls and $0.15 against 12 calls and $1.25!

deepseek v4.1 flash ran all four three.js scenes for $0.15. gemini 3.8 flash needed $1.25 we put the two models on one job: a self-contained html page that renders an animated 3d scene with three.js – a storm at sea, a spiral galaxy, a tornado over farmland, a tesla coil on a dark table. same prompt - calls until all four scenes were usable #1 deepseek v4.1 flash – 6 #2 gemini 3.8 flash – 12 - time #1 deepseek v4.1 flash – 13m 57s #2 gemini 3.8 flash – 32m 02s - cost #1 deepseek v4.1 flash – $0.15 #2 gemini 3.8 flash – $1.25 observations: • gemini blew out the exposure on 4 of 5 first attempts – a white blob where the tesla coil should be. the prompt had a paragraph on exposure. it did not help. • both models overcorrect in the fix loop: "too bright" becomes "almost black". round two is where it lands. same four scenes, 6 calls and $0.15 against 12 calls and $1.25!

thehype.

62,603 次观看 • 11 天前

Built an AI maturity assessment. It's designed to answer... • how AI-native your team really is • where the bottlenecks are • what good looks like • and what to do next We turned the AI transformation model into something practical you can actually use. Takes about 3 minutes. Chat with it and it gives back: • a detailed AI maturity assessment • strengths and gaps across your workflows • practical recommendations • a downloadable report you can share internally The goal wasn’t to make a gimmicky quiz (please give me direct feedback on how to improve). It’s meant to be a practical diagnostic for how modern teams are actually adopting AI at work. Give it shot!

Built an AI maturity assessment. It's designed to answer... • how AI-native your team really is • where the bottlenecks are • what good looks like • and what to do next We turned the AI transformation model into something practical you can actually use. Takes about 3 minutes. Chat with it and it gives back: • a detailed AI maturity assessment • strengths and gaps across your workflows • practical recommendations • a downloadable report you can share internally The goal wasn’t to make a gimmicky quiz (please give me direct feedback on how to improve). It’s meant to be a practical diagnostic for how modern teams are actually adopting AI at work. Give it shot!

Hurley

33,232 次观看 • 4 个月前