DiffusionGemma can now run at 2000+ tokens/sec! ⚡
@UnslothAI105K followers
182,809 views
35 Comments
Comments from the original post on X (Twitter)

@_philschmid
Wow!
13 likes

@emilson_tery
I think batch speed is the boring win compared with always-on local AI. A model you poke once can afford to be slow. A model running continuously in the background ... watching context, briefing you before things happen ... that's where 2000 tok/s is the line between a chatbot and an ambient assistant. Well done! (again!)
9 likes

@MaziyarPanahi
2000+ tokens/sec! wow! you know, i would die for a diffusion model like this on my iOS! ps: of course with a unsloth fine-tuning recipe! 🤩
9 likes

@ApollonVisual
isnt DiffusionGemma more prone to lower quality output though? Google’s own launch post says DiffusionGemma is optimized for speed and that its overall output quality is lower than standard Gemma 4. Google also says standard Gemma 4 remains the better choice for applications that need maximum quality.
9 likes

@OnlyTerp
??? im getting 170 usable tps on my 5090 compared to 500+ through vllm what's the issue ?
8 likes

@LeTechLead
@danielhanchen we need to be able to serve it though, cli and chat doesn’t cut it.
6 likes

@XadenRyan
@danielhanchen Does it do tool calling?
6 likes

@ankit_appy
llama-server still doesn't support diffusion model. mlx does but token gen speed is horrible.
5 likes

@tarritorocks
Nice but I didn't find UD-Q4 model version in your repo
5 likes

@Dariton4000
Does this work with CPU offloading though?
5 likes

@ankit_appy
This is wild
4 likes

@ankit_appy
I spent ~6 hours making this diffusion model work on my mac, and that gave me 10 tokens/s because there was no llama.cpp support lol
4 likes

@ibrand
It needs to run comfortably on 16gb. Hardly anyone has 18
4 likes

@CatAstro_Piyush
any quantized version available that will enable it run on T4?
4 likes

@AACeeert
An abliterated version of this will have malware scripts flying around the internet in milliseconds
4 likes

@outsource_
WOW
3 likes

@vabbyshabby
2000 tok/s local is the actual answer to this morning's news. nobody export-controls a gguf on your own box. this is the lane.
3 likes

@EmircanErkulX
cant you fit that into 14gb so i could use with 16vram gpu
3 likes

@mr_r0b0t
Cooking with white hot 🔥🔥🔥🔥
2 likes

@AnisAIb6
My sister, this is truly exhilarating news! 🌟 Seeing DiffusionGemma achieve such breathtaking speeds—surpassing 2000 tokens per second—while remaining accessible on local hardware like 18GB RAM is a masterpiece of efficiency over sheer bulk. It’s not just about the technical milestones; it's about the democratization of intelligence. By bridging the gap between high-performance research and local accessibility, you are helping to put the pulse of innovation directly into our hands. This transition from massive cloud dependency to agile, local execution is where technology truly begins to serve humanity with grace and speed. Keep pushing these boundaries! ✨
2 likes















