Video wird geladen...
Video konnte nicht geladen werden
I used Opus 5.5 to formally verify the Claude Agent SDK using Lean. A couple short prompts = 16 PRs fixing various bugs and race conditions. Video attached. TLA+ also works well. I sometimes combine Lean and TLA+ to look for issues around data flow, concurrency, and state mgmt.... show more
1,945,525 Aufrufe • vor 2 Tagen •via X (Twitter)
39 Kommentare

Opus made an infographic

it is!

@mmatthias You should try @quint_lang TLA+ with more modern syntax

Opus 5.5 is by far a model no one expected to be this good in so many various fields personally love its animation capability and the fact it's faster for tasks that don't require too much thinking from the model

Yea I've personally gotten tons of leverage from using this, especially when its not just an app but a larger system with multiple apps talking to each other. I barely passed my formal verification class in school and I only know about TLA+ from talks. Thanks for sharing your prompts for this.

btw, I see high quota usage from general-purpose subagent, and almost no docs on how to tune it. are there any guidelines for that?

Ask Claude to use subagents less, or ask it to disable subagents in your permissions

how often do you use subagents and what are your use cases for them? I always thought it is nice to spawn them to offload main orchestrator session

@bcherny The split that works for me isn't task size, it's whether I need the reasoning. Fan out a search where I only want the answer. Keep it in the main session when I'd want to argue with a step, because a subagent hands back a conclusion already made and you can't reopen it.

@bcherny well I have two use cases: * personal - I am on 20x subscription and it is smooth as long as I dont get lazy with prompting and rational use of models and effort

@bcherny and the second one is enterprise subscription for my work and I have a tendency to become lazy and just create one session where I dump prompts for the model to fan out and fix in bulk

@bcherny and on enterprise it is pretty hard to keep quota clean for the whole month, I have to sit on sonnet low effort most of the time

Using Opus 5.5 in Rocq right now! Great application.

Can I please join the team-lunch channel? I'm hungry

For race conditions, I’d treat “does the model match the code?” as a separate review from “does the proof check?” That’s the gap I’d be most careful about before calling something verified.

I get a sense that Boris is having a lot of fun (genuinely!) at work these days. This is a truly a great time.

Even just using it today, It’s a massive difference from Opus 5! 👏🏻

cool idea. no idea what the f it does. in the future i will ask claude to formally verify my agents using physics simulators

Have you tried quint?

Whoa

Looks interesting, but what was a specific need to TLA+ here, which parts?

I've found that TLA+ is great for finding concurrency issues and races

what is lean

I’ve been bullish on formal verification x agentic coding for a while now!

proofs beat vibes for agent code

wdym "future of coding?" I thought coding was solved? 😅

a couple short prompts and 16 bug fixes. my to-do list is feeling very attacked right now

Can we formally verify Lean's kernel first? Have you pointed Fable and Opus at it to look for bugs? 😅

This is how vibe coding grows up. Most people building with Claude can't fully check the code they ship, me included. If the model can prove its own code is right, that gets a lot less scary.

The future of AI is for artists- not scientists.

race conditions were the easy half. the part that ate me was the verifier running in a different env than the agent. did your verifier share the agent's filesystem view, or its own sandbox

16 PRs from a couple prompts is wild. It's crazy what tools like Opus and Claude can dig up.

bend @VictorTaelin

flawless prompting

Formal methods become much more approachable when an agent can translate a codebase into Lean or TLA+ models. The key benchmark is the counterexample loop: time to a reproducible trace, fix, and re-check, with false positives and model omissions reported.

you have finally fixed managing bulk prs 🙏

couple prompts, 16 PRs. totally normal Tuesday.

16 PRs from a couple prompts is wild. formal verification suddenly feels usable for ordinary teams who never had a Lean expert sitting nearby

Proof is the only receipt an agent should leave. Opus drafting Lean checks for its own SDK is the timeline flipping: the runtime starts auditing the hands that hold it.
