DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

Oh my God, bro I have been refreshing my browser all night long waiting for you to say this

Local multimodal is leaving the lab bench.

Can we have it in Unsloth?

My bro, go look through your PR‘s. There is one there that stops vision from fucking up the cache. It’s a simple PR and it should work for both models, but it was written for the 5.3 setup.

shipping Metal, CUDA and ROCm together on day one is the notable part, most local-inference projects keep CUDA solid and let the other two lag for months. usually means the quantization/kernel abstraction is actually clean, not just that the model fits.

will it also run on a single dxg spark?

Crazy Speed….. Thank You Sir~~~

Hero

Cool! I'd love to play with similar optimisations on smaller models. I replicated with qwen3.8:27B on an M4 max (TG 21t/s) and a 3.5:9B on a Steam Deck (11t/s) and while answers are already more than decent (the 9b even got Blue Marble!) they could benefit from a speed-up

I can't decide between running GLM-5.3-Flash and DeepSeek-V4-Flash on the 512 GB M3 Ultra now 😅

Do you sleep?

Metal and CUDA implementations sharing the same release window usually means the edge cases are in the memory alignment layer.

👍

I kept the vision demo on the API and called the Mac a spare. tools.md now: one local Metal run on the M5 before I paste a cloud screenshot. Last tests, then I ship.

Is it quantized model ?

i ran it on my custom quant and had to fix 2 small things (PR #940 and kind-of also #941) to get images working in pi.

Odd to see Metal, CUDA and ROCm all finished together. I'd guess ROCm was the last one to pass tests. Fast vision inference on an M5 Max stands out most. Wonder what tok/s it's hitting on vision.

Sei un genio

is the release full vision weights or an adapter on top of the text checkpoint?

棒

Odd to see Metal, CUDA and ROCm all finished together. I'd guess ROCm was the last one to pass tests. Fast vision inference on an M5 Max stands out most. Wonder what tok/s it's hitting on vision.

🎉

Excellent 👌

what speed did u achieved? Wouldn't it be just better to use it?