Right now, when you send a query to an LLM, it gets decrypted on the server. The LLM sees your data in plain text. Prof. Ajay Joshi (BU, CipherSonic AI ) on fully homomorphic encryption, which may be key for the future of AI privacy: how we can compute... on data without ever decrypting it. The catch: it's a brutally memory-bound workload. Exactly the bottleneck wafer-scale was built to solve.show more

@CipherSonicAI I know a reasonable amount about LLMs, and a ton about homomorphic encryption, and I cannot believe that this will ever be computationally feasible. The slowdown is simply too large.

Watch the full interview:

In conversation with @MilksandMatcha for Big Chip Club. Produced by @alyciazcary.

@CipherSonicAI have you seen this @cerebras @CipherSonicAI

Pretty amazing they managed to revive old technology that came with a 100000x performance penalty to be just twice as slow to have it encrypted. I wouldn't mind cutting performance in half if I could trust that all of my data and/or requests could not be decrypted. especially as it relates to really sensitive data.

@CipherSonicAI We will probably get a colony in Mars before homomorphic encryption gets solved (if ever)

Since I work in the medical field, we can't be sending patient data to the the llm, we developed a different mechanism that uses code instead of text to interface with the environment. I built-a ontology-based programming language that thinks and remembers in code. Patient data never gets sent to the LLM at all. It's open source, I invite anybody to read more about it:

@CipherSonicAI .. why not just sell to businesses directly, so they don't need to trust third parties to do inference for them?

@CipherSonicAI FHE is not possible at all due to the computation complexity. Order of 1000s magnitude.

@CipherSonicAI What you are asking for is a contradiction in terms. If you want security you should be running your models locally on your own hardware, not demanding impossibilities.

@CipherSonicAI Wait so the LLM literally sees everything in plain text right now, never thought about it that way

@CipherSonicAI

@CipherSonicAI Privacy-preserving AI is one of the most important challenges ahead. Fully homomorphic encryption could be a game changer if the performance hurdles can be solved.

@CipherSonicAI i will be honest i did'nt like it at all. atleast build something... giving away 2 free models and i don't know what your premium look like, creating hype bro we get all that from open router, open code, claude code its just a hype

@CipherSonicAI I wonder how this is different from TEE models

@CipherSonicAI cool concept

@CipherSonicAI This could solve a lot of worries which drive people to want to own their infra.

@CipherSonicAI Confidentiality and provenance are two different guarantees. FHE keeps your data private from the server. SiGR proves what the model did with it. Neither replaces the other. Would love to see Hive signing the compute that FHE protects.

Check out We support an encrypted AI assistant that uses Llama 3.2-90B. Access to encrypted inference using Llama 3.2-90B is also available through APIs.

@CipherSonicAI Privacy is becoming one of the biggest challenges in AI. Advancements like fully homomorphic encryption could be a major step toward enabling secure, privacy-preserving AI without compromising performance. Excited to see this research continue to evolve. 🔐🤖

@CipherSonicAI FHE memory overhead is brutal, easily 100x expansion. Wafer-scale actually fits, regular GPUs just choke on cache thrashing before the math finishes.

@CipherSonicAI I do not want gay ANYTHING with my query!

@CipherSonicAI @grok So is Cerebras working on TEEs or what? FHE will take too long to make performant...

@CipherSonicAI We should respect the elastic contexting with llms for continued coherent conversations.