正在加载视频...
视频加载失败
Before Fable got released (and pulled) Mozilla was quietly testing Claude Mythos against Firefox's 10M line codebase. The result? Over 400 security bugs fixes, including ones that had been hiding in the codebase for over a decade. Brian Grinstead, distinguished engineer at Mozilla, walked me through the agentic bug-finding... show more
31 条评论

Ty ty to our amazing sponsors: 💼 @WorkOS - Make your app enterprise ready today 🏆 @MetaviewAI - the agentic recruiting platform for winning teams 🎤 Listen everywhere: ▶️ YouTube: 🎧 Spotify: 🍎 Apple:

Full breakdown on the @chatprd blog:

@hnshah @mozilla @bgrins Watching this for Brian’s loop pattern ideas!

@mozilla @bgrins Insane reminder that most “AI breakthroughs” are really just better agent loops + verification, not magic models.

This is exactly why deterministic enforcement matters more than ever. Even when agents find real value (like the 400+ bugs at Mozilla), the risk of them making destructive changes in production codebases is real — especially with frontend stacks using Cursor/Claude. I built Nemesis Defender for this: Pretool Hook (exit(2) before execution) + Iron Dome scanner + eBPF kernel protection. 100% local, tamper-proof, calibrated for Next.js/React/TS. Project + demo: Would love your thoughts on agent harnesses vs hard boundaries, Claire.

@mozilla @bgrins friend developed a faster code search tool to make this easier called src-nav which does things grep cannot

@mozilla @bgrins Ma’am I know I probs come off as a weird fangirl but these episodes are so informative and helpful - thank you for doing what you do! 😍

@mozilla @bgrins finding 400 bugs is easier than proving the patches don't add regressions

@mozilla @bgrins To point a model deterministically where duplications are in large codebases, dupehound does this really well, superfast and open-source.

@mozilla @bgrins the 'you can build this in an afternoon' part is either the most motivating or most humbling thing ive read all week no in between tbh 😭

@mozilla @bgrins Insane

@mozilla @bgrins What's the "lie to the agent" part about, curious how that helped with false positives

@mozilla @bgrins 400 fixes sounds great but id wanna know how many of those got properly reviewed before merge agent found it doesnt mean a human understood it

@mozilla @bgrins The "50% setup" line is the whole lesson. Everyone argues about which model. The model is the cheap part now. The harness is the skill: a tight goal/loop and a verifier that kills false positives. Same model, two setups, completely different output.

@mozilla @bgrins The "afternoon to write your own harness" line is the headline. If that harness plus a verifier yields 400 confirmed bugs, the result is a property of the setup, not of Mythos. Any capable model fits that slot. The scarce part was never the model, it was the scaffolding.

@mozilla @bgrins I wanna know what lies to tell!

@mozilla @bgrins 十年老 bug 被翻出来,这就有点狠了

@mozilla @bgrins Yes, agent power stems from the model and harness, like Schumacher and a Ferrari.

@mozilla @bgrins Mythos was able to find some really gnarly bugs too. Impressive considering Firefox has some extremely bright engineers. As competent the human mind is, it's simply not as good a repository of structured information, as LLMs, which is probably why LLMs excel at this.

@mozilla @bgrins The only gates to a deeply integrated, self directed agentic environment are careful reading and well articulated questions.

@mozilla @bgrins lying to the agent to get better results is the part im stuck on that sounds wild and i need the whole explanation NOW

@mozilla @bgrins not sure 400 fixes is the headline here, curious what the validation loop actually looked like. at 10M lines, how many of those patches had a human review before merge?

@mozilla @bgrins decade old bugs just vibing in the codebase like a tenant who stopped paying rent in 2014 but nobody checked 💀

@mozilla @bgrins the load on newly released models is going to be insane from now on who knows how long it will be up !!!

@mozilla @bgrins the model isn't the product. the loop around the model is where the leverage hides.

In the agentic security bug pipeline for Firefox, how did you handle prioritizing and sequencing work across the codebase (or potential issues), respecting compute costs, dependencies, or human review bandwidth — and did the harness integrate directly with existing bug tracking/backlog tools, or was that kept fully human-driven?

@mozilla @bgrins 499 isnt much though

@mozilla @bgrins Don’t you think it’s wierd that Anthropic have it to Mozilla for a browser rather than to the US govern to secure its systems? Wierd.

@mozilla @bgrins I feel this is like tech debt and you have to get to it sooner or later. But agents make it possible to do so fast and well, that it is potentially not as big of a deal as before. I see companies releasing their stacks in rust so frequently now.

@mozilla @bgrins this is massive. the real unlock isn't just testing your model against a big codebase, it's what that says about the state of code maintenance and how we need tools like claude to catch up with our own sloppiness

@mozilla @bgrins the 50/50 split is the real takeaway people keep crediting the model but the harness doing half the work is not a small detail
