I am happy to share FlashProve: our performant open-source prototype for provable inference. We believe that this has significant implications for security, pacing AI development, integrity guarantees, and more. FlashProve allows a provider to commit to model weights, and then prove that replies originate from the committed model. The... proof leaks no information about the weights, and the verifier can efficiently verify it on consumer hardware. Our library is performant, proving a 400B parameter MoE model at 100 tok/s. We have open-sourced the associated code and Lean formalization. We hope that it catalyzes further progress in verification of inference, and look forward to seeing it.show more

FlashProve is a commit-and-prove succinct zero-knowledge argument. The provider commits to the model weights once, and sends the commitment to clients. When a client submits a prompt, the prover attaches a short proof, and the client (on consumer-hardware) can check the proof against the commitment.  The proof cryptographically guarantees that the provider used the model claimed to generate the response, and the provider is assured that it does not leak information about the weights. Further, the proof is succinct: it is short and the verifier can efficiently check it on consumer hardware.

To build FlashProve, we rely on battle-tested cryptography: interactive oracle proofs and cryptographic hash functions. An interactive oracle proof is an interactive proof system, in which a prover sends long oracle messages which the verifier can query. Using Merkle commitments, we replace the oracles with cryptography, and we use Fiat-Shamir to remove interactivity. These succinct arguments have many desirable properties: minimal assumptions, post-quantum security, and fast provers. We tailor our interactive oracle proof to machine learning computation, and achieve succinctness via fold and hopping, an adaptation of the techniques in Zook.

To achieve great performance, we had to complement this with a large amount of engineering. Safe to say, Claude was very happy to help! We prove the entire model streaming on a single GPU (which cannot even fit the weights!). For a 400B mixture-of-experts model, we can prove at more than 100tok/s, just two orders of magnitudes slower than running the model natively!  We are aware of a number of further optimizations, so expect the number to improve!

We have (many) more details in the blog posts and the repositories. We also expect to post a (human generated) paper explaining the protocol in more details soon, so stay tuned for that one as well. FlashProve wouldn’t have been possible without the excellent work of everyone involved: @RoyRinberg , Alex, Ale, @miranda__christ, Rahul, @transmissions11 and @gwberg95 (and Claude). It was a lot of fun!

Very cool, congrats! @GiacomoFenzi how do you handle noise in its various forms? Is my understanding correct that you make the model deterministic, i.e., no race conditions and no floating point issues? How about other input that may sneak in: time, tool calls, prompt grinding?

Yes! Currently for the prototype we quantize the model to make it deterministic. We have a few other tricks in store for the general case, that we hope to share soon.

congrats man!! Awesome work.

Thank you!

Congrats! Do you expects further performance improvements from the protocol or the GPU prover implementation side?

Thank you! Both! I think there is juice to squeeze in many places

Great to know. Cann’t wait to build on top of it!

oooo

@GiacomoFenzi goated

gotta cook some time 🧑‍🍳

wanted to confirm if i am understanding this part correctly: the models are being quantized to integer only to get determinism in matrix multiplication (due to arithmetization being over only integer fields?) and that there is another check to ensure that error introduced due to integer quantization is not being used as cover for breaking soundness?

Congrats on shipping FlashProve, Giacomo! Really cool to see this open-sourced. #AI #OpenSource

@claucece Wow, that's so good!

@claucece Thank you! Very very happy ;)

Awesome stuff! Truly incredible to see zk proofs to secure llm!

Thank you!

Lo chef

🧑‍🍳🧑‍🍳🧑‍🍳

Your report says FlashProve generates proofs after replies are served, by recomputing transcripts on separate hardware. A valid proof shows what the committed model would output. It doesn’t establish which model actually served those tokens. How can computational consistency prove the response’s origin?

didn't think my laptop could do this

nice work. we just open sourced our private, sovereign ZK AI protocol for ethereum

One inference proved is a big step. Agent runs are dozens of calls with tool results feeding back in between. Curious how you see per-call proofs composing into evidence of the whole run, including the non-model steps a verifier still has to take on trust.

provable inference only matters if the prover cannot swap the model mid-run

the sampling part is where i'd poke. with temperature above zero, a provider could swap in a cheaper model and claim a lucky sample unless the seed is committed too. is the randomness bound into the proof, or is this greedy decoding only?