Loading video...

Video Failed to Load

Go Home

I am happy to share FlashProve: our performant open-source prototype for provable inference. We believe that this has significant implications for security, pacing AI development, integrity guarantees, and more. FlashProve allows a provider to commit to model weights, and then prove that replies originate from the committed model. The...

12,157 views • 3 days ago •via X (Twitter)

28 Comments

Giacomo Fenzi's profile picture
Giacomo Fenzi3 days ago

FlashProve is a commit-and-prove succinct zero-knowledge argument. The provider commits to the model weights once, and sends the commitment to clients. When a client submits a prompt, the prover attaches a short proof, and the client (on consumer-hardware) can check the proof against the commitment.  The proof cryptographically guarantees that the provider used the model claimed to generate the response, and the provider is assured that it does not leak information about the weights. Further, the proof is succinct: it is short and the verifier can efficiently check it on consumer hardware.

Giacomo Fenzi's profile picture
Giacomo Fenzi3 days ago

To build FlashProve, we rely on battle-tested cryptography: interactive oracle proofs and cryptographic hash functions. An interactive oracle proof is an interactive proof system, in which a prover sends long oracle messages which the verifier can query. Using Merkle commitments, we replace the oracles with cryptography, and we use Fiat-Shamir to remove interactivity. These succinct arguments have many desirable properties: minimal assumptions, post-quantum security, and fast provers. We tailor our interactive oracle proof to machine learning computation, and achieve succinctness via fold and hopping, an adaptation of the techniques in Zook.

Giacomo Fenzi's profile picture
Giacomo Fenzi3 days ago

To achieve great performance, we had to complement this with a large amount of engineering. Safe to say, Claude was very happy to help! We prove the entire model streaming on a single GPU (which cannot even fit the weights!). For a 400B mixture-of-experts model, we can prove at more than 100tok/s, just two orders of magnitudes slower than running the model natively!  We are aware of a number of further optimizations, so expect the number to improve!

Giacomo Fenzi's profile picture
Giacomo Fenzi3 days ago

We have (many) more details in the blog posts and the repositories. We also expect to post a (human generated) paper explaining the protocol in more details soon, so stay tuned for that one as well. FlashProve wouldn’t have been possible without the excellent work of everyone involved: @RoyRinberg , Alex, Ale, @miranda__christ, Rahul, @transmissions11 and @gwberg95 (and Claude). It was a lot of fun!

Jens Groth's profile picture
Jens Groth2 days ago

Very cool, congrats! @GiacomoFenzi how do you handle noise in its various forms? Is my understanding correct that you make the model deterministic, i.e., no race conditions and no floating point issues? How about other input that may sneak in: time, tool calls, prompt grinding?

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

Yes! Currently for the prototype we quantize the model to make it deterministic. We have a few other tricks in store for the general case, that we hope to share soon.

Georgios Konstantopoulos's profile picture
Georgios Konstantopoulos3 days ago

congrats man!! Awesome work.

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

Thank you!

drouyang's profile picture
drouyang3 days ago

Congrats! Do you expects further performance improvements from the protocol or the GPU prover implementation side?

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

Thank you! Both! I think there is juice to squeeze in many places

drouyang's profile picture
drouyang2 days ago

Great to know. Cann’t wait to build on top of it!

Princess Vegeta's profile picture
Princess Vegeta3 days ago

oooo

pietro's profile picture
pietro3 days ago

@GiacomoFenzi goated

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

gotta cook some time 🧑‍🍳

Soubhik Deb's profile picture
Soubhik Deb2 days ago

wanted to confirm if i am understanding this part correctly: the models are being quantized to integer only to get determinism in matrix multiplication (due to arithmetization being over only integer fields?) and that there is another check to ensure that error introduced due to integer quantization is not being used as cover for breaking soundness?

Robert Corbin | Shadowfetch Studio | AIpeoplewiki's profile picture
Robert Corbin | Shadowfetch Studio | AIpeoplewiki3 days ago

Congrats on shipping FlashProve, Giacomo! Really cool to see this open-sourced. #AI #OpenSource

osdnk's profile picture
osdnk2 days ago

@claucece Wow, that's so good!

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

@claucece Thank you! Very very happy ;)

Loris's profile picture
Loris3 days ago

Awesome stuff! Truly incredible to see zk proofs to secure llm!

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

Thank you!

Francesco Intoci's profile picture
Francesco Intoci2 days ago

Lo chef

Giacomo Fenzi's profile picture
Giacomo Fenzi2 days ago

🧑‍🍳🧑‍🍳🧑‍🍳

AndrewJoes's profile picture
AndrewJoes2 days ago

Your report says FlashProve generates proofs after replies are served, by recomputing transcripts on separate hardware. A valid proof shows what the committed model would output. It doesn’t establish which model actually served those tokens. How can computational consistency prove the response’s origin?

Shez Malik's profile picture
Shez Malik2 days ago

didn't think my laptop could do this

Zeko Labs's profile picture
Zeko Labs2 days ago

nice work. we just open sourced our private, sovereign ZK AI protocol for ethereum

JB's profile picture
JB2 days ago

One inference proved is a big step. Agent runs are dozens of calls with tool results feeding back in between. Curious how you see per-call proofs composing into evidence of the whole run, including the non-model steps a verifier still has to take on trust.

0xJoshAlpha's profile picture
0xJoshAlpha2 days ago

provable inference only matters if the prover cannot swap the model mid-run

Shavaiz Tariq's profile picture
Shavaiz Tariq3 days ago

the sampling part is where i'd poke. with temperature above zero, a provider could swap in a cheaper model and claim a lucky sample unless the seed is committed too. is the randomness bound into the proof, or is this greedy decoding only?

Related Videos

Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

Dustin

688,855 views • 8 months ago

America is about to lose the AI race, and it will not happen at the frontier. It will happen at the floor. Everyone is watching who ships the smartest model. The actual war is over the 80% of tokens nobody posts about: the routine inference that quietly runs the world. On current trajectory, that fight is already lost. Watch what people do, not what they say. Coinbase just defaulted its own engineers off frontier models onto open weights and cut AI spend nearly in half while usage kept climbing. Even NVIDIA runs a closed frontier model as an orchestrator and pushes the volume to its own open weights. The frontier is becoming a router. The volume goes open. That part is settled. Here is the part that should terrify Washington: the only credible open tier today is Chinese. GLM. Kimi. And the US answer is to tighten export controls and freeze its own labs in place, as if you can embargo a file that is already downloaded, or price-match free. So China hands the Global South Huawei hardware and free open models, and a generation in Africa and Southeast Asia learns to reason through a model that will not tell them what happened at Tiananmen Square. That is not a cost story. That is influence through inference. You do not have to win hearts and minds when you supply the mind. Open source is not a nice-to-have for America. It is the whole ballgame for the 80%, and right now the US is barely on the field. We need American open weights. Not eventually. Now.

Ben Pouladian

71,722 views • 4 months ago