Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

62,060 görüntüleme • 2 ay önce •via X (Twitter)

24 Yorum

Context Bandit profil fotoğrafı
Context Bandit2 ay önce

Oh my God, bro I have been refreshing my browser all night long waiting for you to say this

Liam Nerd profil fotoğrafı
Liam Nerd2 ay önce

Local multimodal is leaving the lab bench.

unbug profil fotoğrafı
unbug2 ay önce

Can we have it in Unsloth?

Context Bandit profil fotoğrafı
Context Bandit2 ay önce

My bro, go look through your PR‘s. There is one there that stops vision from fucking up the cache. It’s a simple PR and it should work for both models, but it was written for the 5.3 setup.

saietta profil fotoğrafı
saietta2 ay önce

shipping Metal, CUDA and ROCm together on day one is the notable part, most local-inference projects keep CUDA solid and let the other two lag for months. usually means the quantization/kernel abstraction is actually clean, not just that the model fits.

Antonio Pellegrini profil fotoğrafı
Antonio Pellegrini2 ay önce

will it also run on a single dxg spark?

KimSunIl profil fotoğrafı
KimSunIl2 ay önce

Crazy Speed….. Thank You Sir~~~

Norbert Schmidt profil fotoğrafı
Norbert Schmidt2 ay önce

Hero

Davide Eynard profil fotoğrafı
Davide Eynard2 ay önce

Cool! I'd love to play with similar optimisations on smaller models. I replicated with qwen3.8:27B on an M4 max (TG 21t/s) and a 3.5:9B on a Steam Deck (11t/s) and while answers are already more than decent (the 9b even got Blue Marble!) they could benefit from a speed-up

Xaden Ryan profil fotoğrafı
Xaden Ryan2 ay önce

I can't decide between running GLM-5.3-Flash and DeepSeek-V4-Flash on the 512 GB M3 Ultra now 😅

AIGottaStayFly profil fotoğrafı
AIGottaStayFly2 ay önce

Do you sleep?

BullBear.News profil fotoğrafı
BullBear.News2 ay önce

Metal and CUDA implementations sharing the same release window usually means the edge cases are in the memory alignment layer.

ICSR profil fotoğrafı
ICSR2 ay önce

👍

Nandan Priyadarshi profil fotoğrafı
Nandan Priyadarshi2 ay önce

I kept the vision demo on the API and called the Mac a spare. tools.md now: one local Metal run on the M5 before I paste a cloud screenshot. Last tests, then I ship.

Gilbert profil fotoğrafı
Gilbert2 ay önce

Is it quantized model ?

Andreas Petersson profil fotoğrafı
Andreas Petersson2 ay önce

i ran it on my custom quant and had to fix 2 small things (PR #940 and kind-of also #941) to get images working in pi.

_alphashark_ profil fotoğrafı
_alphashark_2 ay önce

Odd to see Metal, CUDA and ROCm all finished together. I'd guess ROCm was the last one to pass tests. Fast vision inference on an M5 Max stands out most. Wonder what tok/s it's hitting on vision.

Bernardo Mascellani profil fotoğrafı
Bernardo Mascellani2 ay önce

Sei un genio

Rimas profil fotoğrafı
Rimas2 ay önce

is the release full vision weights or an adapter on top of the text checkpoint?

MobaiLabs | 墨白实验室 profil fotoğrafı
MobaiLabs | 墨白实验室2 ay önce

棒

_alphashark_ profil fotoğrafı
_alphashark_2 ay önce

Odd to see Metal, CUDA and ROCm all finished together. I'd guess ROCm was the last one to pass tests. Fast vision inference on an M5 Max stands out most. Wonder what tok/s it's hitting on vision.

Anand C. Patel, MD MS profil fotoğrafı
Anand C. Patel, MD MS2 ay önce

🎉

marquesa 🇵🇹🇫🇷🇪🇺 profil fotoğrafı
marquesa 🇵🇹🇫🇷🇪🇺2 ay önce

Excellent 👌

balega_dev profil fotoğrafı
balega_dev2 ay önce

what speed did u achieved? Wouldn't it be just better to use it?

Benzer Videolar