Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

62,060 Aufrufe • vor 2 Monaten •via X (Twitter)

24 Kommentare

Profilbild von Context Bandit
Context Banditvor 2 Monaten

Oh my God, bro I have been refreshing my browser all night long waiting for you to say this

Profilbild von Liam Nerd
Liam Nerdvor 2 Monaten

Local multimodal is leaving the lab bench.

Profilbild von unbug
unbugvor 2 Monaten

Can we have it in Unsloth?

Profilbild von Context Bandit
Context Banditvor 2 Monaten

My bro, go look through your PR‘s. There is one there that stops vision from fucking up the cache. It’s a simple PR and it should work for both models, but it was written for the 5.3 setup.

Profilbild von saietta
saiettavor 2 Monaten

shipping Metal, CUDA and ROCm together on day one is the notable part, most local-inference projects keep CUDA solid and let the other two lag for months. usually means the quantization/kernel abstraction is actually clean, not just that the model fits.

Profilbild von Antonio Pellegrini
Antonio Pellegrinivor 2 Monaten

will it also run on a single dxg spark?

Profilbild von KimSunIl
KimSunIlvor 2 Monaten

Crazy Speed….. Thank You Sir~~~

Profilbild von Norbert Schmidt
Norbert Schmidtvor 2 Monaten

Hero

Profilbild von Davide Eynard
Davide Eynardvor 2 Monaten

Cool! I'd love to play with similar optimisations on smaller models. I replicated with qwen3.8:27B on an M4 max (TG 21t/s) and a 3.5:9B on a Steam Deck (11t/s) and while answers are already more than decent (the 9b even got Blue Marble!) they could benefit from a speed-up

Profilbild von Xaden Ryan
Xaden Ryanvor 2 Monaten

I can't decide between running GLM-5.3-Flash and DeepSeek-V4-Flash on the 512 GB M3 Ultra now 😅

Profilbild von AIGottaStayFly
AIGottaStayFlyvor 2 Monaten

Do you sleep?

Profilbild von BullBear.News
BullBear.Newsvor 2 Monaten

Metal and CUDA implementations sharing the same release window usually means the edge cases are in the memory alignment layer.

Profilbild von ICSR
ICSRvor 2 Monaten

👍

Profilbild von Nandan Priyadarshi
Nandan Priyadarshivor 2 Monaten

I kept the vision demo on the API and called the Mac a spare. tools.md now: one local Metal run on the M5 before I paste a cloud screenshot. Last tests, then I ship.

Profilbild von Gilbert
Gilbertvor 2 Monaten

Is it quantized model ?

Profilbild von Andreas Petersson
Andreas Peterssonvor 2 Monaten

i ran it on my custom quant and had to fix 2 small things (PR #940 and kind-of also #941) to get images working in pi.

Profilbild von _alphashark_
_alphashark_vor 2 Monaten

Odd to see Metal, CUDA and ROCm all finished together. I'd guess ROCm was the last one to pass tests. Fast vision inference on an M5 Max stands out most. Wonder what tok/s it's hitting on vision.

Profilbild von Bernardo Mascellani
Bernardo Mascellanivor 2 Monaten

Sei un genio

Profilbild von Rimas
Rimasvor 2 Monaten

is the release full vision weights or an adapter on top of the text checkpoint?

Profilbild von MobaiLabs | 墨白实验室
MobaiLabs | 墨白实验室vor 2 Monaten

棒

Profilbild von _alphashark_
_alphashark_vor 2 Monaten

Odd to see Metal, CUDA and ROCm all finished together. I'd guess ROCm was the last one to pass tests. Fast vision inference on an M5 Max stands out most. Wonder what tok/s it's hitting on vision.

Profilbild von Anand C. Patel, MD MS
Anand C. Patel, MD MSvor 2 Monaten

🎉

Profilbild von marquesa 🇵🇹🇫🇷🇪🇺
marquesa 🇵🇹🇫🇷🇪🇺vor 2 Monaten

Excellent 👌

Profilbild von balega_dev
balega_devvor 2 Monaten

what speed did u achieved? Wouldn't it be just better to use it?

Ähnliche Videos