Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

DwarfStar with vision support for GLM 5.3 running on a MacBook m5max computer. The vision support is *not* quantized for max precision for now.

antirez's profile picture

antirez

81,922 subscribers

22,664 görüntüleme • 1 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

24 Yorum

Ivan Fioravanti ᯅ profil fotoğrafı
Ivan Fioravanti ᯅ1 ay önce

Amazing! Thanks for this 🙏

Liam Nerd profil fotoğrafı
Liam Nerd1 ay önce

Local multimodal is leaving the server room.

Eduárd Moldován profil fotoğrafı
Eduárd Moldován1 ay önce

How much RAM does that computer provide?

Rich · Atom Tan Studio profil fotoğrafı
Rich · Atom Tan Studio1 ay önce

That speed is impressive, but I wonder how long that will last during a session.

Urban Arson 🔫 profil fotoğrafı
Urban Arson 🔫1 ay önce

Can the Deepseek-V4-Flash model also take advantage of vision support? Or are you doing it in a way that only GLM-5.3 can take advantage of?

brian farley profil fotoğrafı
brian farley1 ay önce

I'm hitting like 40tk/s now with some optimizations to the current branch and much less slow down as context grows. Busy tackling prefill optimizations. Thanks for the update gonna add.

Dejan Marjanović profil fotoğrafı
Dejan Marjanović1 ay önce

Going to run this on Ultra 512GB 💪

Nathan Brake profil fotoğrafı
Nathan Brake1 ay önce

Just following up, it works now after I built and deployed from your GLM vision feature branch. Thank you 👨‍🍳

Anders profil fotoğrafı
Anders1 ay önce

What harness is this?

Dejan Marjanović profil fotoğrafı
Dejan Marjanović1 ay önce

This is Flash, full doesn't work?

Tang Vu profil fotoğrafı
Tang Vu1 ay önce

Non-quantized vision on the M5 Max will likely bottleneck throughput. Have you tested FP8 for the vision encoder? Curious how much precision you'd trade for a speedup there.

Andrew Dubin profil fotoğrafı
Andrew Dubin1 ay önce

Is M5 Max > M4 Max by that much? I get 22 tps on mine.

BullBear.News profil fotoğrafı
BullBear.News1 ay önce

Unquantized vision on an M5 Max means memory bandwidth is doing all the heavy lifting.

korale profil fotoğrafı
korale1 ay önce

Thats huge. Thanks!

0x0- profil fotoğrafı
0x0-1 ay önce

Excited to see you working (and your work) on GLM-5.3-Flash. Curious what your take is on Q2 performance when your opinion is fully formed. Your work is truly an inspiration. Thank you!

Antonio Brundo | AI Architet profil fotoğrafı
Antonio Brundo | AI Architet1 ay önce

Great!

Leeweixuan profil fotoğrafı
Leeweixuan1 ay önce

is the caching for images not working yet or intended?

Andrew Bonstrom profil fotoğrafı
Andrew Bonstrom1 ay önce

Is this live now in ds4 or a tease for a soon to be shipped PR? 😁

Aleksandar Janca profil fotoğrafı
Aleksandar Janca1 ay önce

why leave just the vision half unquantized, does quantizing it break something specific or just seemed safer for now

Arthur N profil fotoğrafı
Arthur N1 ay önce

GLM-5.3-Flash-Q2 It's a golden-hour street scene in Vieux Nice (old town), France, looking down a stepped street (likely the Montée/Rue Sainte-République area near Place Rossetti): - Warm pastel façades (ochre, terracotta, cream) with shuttered windows and wrought-iron ...

Aseem Shrey profil fotoğrafı
Aseem Shrey1 ay önce

What quants ?

will w profil fotoğrafı
will w1 ay önce

Very glad to hear this information.

Nathan Brake profil fotoğrafı
Nathan Brake1 ay önce

Are you reading my mind? I was just hoping for this yesterday evening after I got it running

Sai Prakash profil fotoğrafı
Sai Prakash1 ay önce

Hope @Apple sends you an early M5 Ultra rig when ready - there isn’t a better test bed and example of usage!

Benzer Videolar

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.
1:19

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

antirez

62,060 görüntüleme • 1 ay önce

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.
1:31

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

antirez

319,303 görüntüleme • 3 ay önce

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...
0:39

GLM 5.2 flash Q2 (generated by myself with similar recipe of DS4 Flash of DwarfStar GGUF) running on an M5 Max with 128GB. The Q4 weights also work on tensor parallel inference in two M5 Max systems. Soon on GitHub. Still testing + CUDA + ROCm ...

antirez

29,548 görüntüleme • 1 ay önce

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now *without* DSpark, so this is the min speed you get in practice.
1:29

I'm optimizing tensor parallel execution across two MacBook M5 Max systems using RDMA. This is how fast DeepSeek v4 Flash Exp Vision can run in MXFP4 (full precision) right now *without* DSpark, so this is the min speed you get in practice.

antirez

27,006 görüntüleme • 1 ay önce

GLM 5.3 Max vs Fable 5 vs Qwen 3.8 Max vs Grok 4.6 GLM 5.3 is a massive jump over 5.2. Still not beating Fable 5 for me, but it’s way closer now. And vs Grok 4.6? I’d take GLM 5.3 pretty easily on this test.
0:33

GLM 5.3 Max vs Fable 5 vs Qwen 3.8 Max vs Grok 4.6 GLM 5.3 is a massive jump over 5.2. Still not beating Fable 5 for me, but it’s way closer now. And vs Grok 4.6? I’d take GLM 5.3 pretty easily on this test.

OmedTheVibeCoder

16,648 görüntüleme • 2 ay önce

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
1:24

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

173,075 görüntüleme • 5 ay önce

Here’s a first look at Adobe Premiere running natively on Apple Vision Pro. The visionOS app features a full spatial editing interface with floating panels, support for spatial clips, and depth controls for text. Available now on the App Store.
0:33

Here’s a first look at Adobe Premiere running natively on Apple Vision Pro. The visionOS app features a full spatial editing interface with floating panels, support for spatial clips, and depth controls for text. Available now on the App Store.

Justin Ryan

22,492 görüntüleme • 3 ay önce

I chopped the head off a broken screen'd M1 Max MacBook Pro to become a fancy keyboard with a mobile compute powerhouse for Vision Pro
0:47

I chopped the head off a broken screen'd M1 Max MacBook Pro to become a fancy keyboard with a mobile compute powerhouse for Vision Pro

Brad Lynch

3,156,854 görüntüleme • 2 yıl önce

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.
0:43

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.

Alex Cheema

128,404 görüntüleme • 9 ay önce

I can’t believe something like this could come out of a model with no vision GLM-5.3 rocks
1:20

I can’t believe something like this could come out of a model with no vision GLM-5.3 rocks

0xSero

25,941 görüntüleme • 2 ay önce

inference on your head mistral 7b (4bit quantized) running locally on apple vision pro
0:31

inference on your head mistral 7b (4bit quantized) running locally on apple vision pro

Joseph Semrai

239,991 görüntüleme • 2 yıl önce

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.
4:15

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

antirez

276,710 görüntüleme • 4 ay önce

Maximum support for the Iranian people — combined with maximum pressure on the regime — is the key to #Iran's future. Iranian Crown Prince Reza Pahlavi shared his vision at FDD's event, "Maximum Support: Operationalizing the Other Iran Policy." For more:
1:14

Maximum support for the Iranian people — combined with maximum pressure on the regime — is the key to #Iran's future. Iranian Crown Prince Reza Pahlavi shared his vision at FDD's event, "Maximum Support: Operationalizing the Other Iran Policy." For more:

FDD

98,784 görüntüleme • 1 yıl önce

SNAPCHAT PAID $150,000,000 FOR LOOKSERY - STARTUP IN DEEP LEARNING COMPUTER VISION This 1-hour Stanford lecture on "DL for Computer Vision" will teach you how to build same project from scratch. Bookmark it & watch today. Stanford's full course (19 lectures) on Deep Learning for Computer Vision below ↓
1:02:47

SNAPCHAT PAID $150,000,000 FOR LOOKSERY - STARTUP IN DEEP LEARNING COMPUTER VISION This 1-hour Stanford lecture on "DL for Computer Vision" will teach you how to build same project from scratch. Bookmark it & watch today. Stanford's full course (19 lectures) on Deep Learning for Computer Vision below ↓

Movez

57,869 görüntüleme • 5 ay önce

"We (Democrats) have to provide a vision, and the vision is of a functional democracy. The vision is of a government that is not corrupt. The vision is of a government that is actually focused on solutions for people."
0:25

"We (Democrats) have to provide a vision, and the vision is of a functional democracy. The vision is of a government that is not corrupt. The vision is of a government that is actually focused on solutions for people."

George Conway ⚖️🇺🇸

41,301 görüntüleme • 5 ay önce

Looking for a better local model that runs on a MacBook Air? Our latest update brings full Gemma 4 12B support to Google AI Edge Gallery app on Mac. ✨ Multimodal: Vision & Audio inputs 🎛️ Configurable vision token budgets ⚡️ MTP for blazing-fast inference 💻 Optimized to run on 16GB Macs! Download it today:
0:25

Looking for a better local model that runs on a MacBook Air? Our latest update brings full Gemma 4 12B support to Google AI Edge Gallery app on Mac. ✨ Multimodal: Vision & Audio inputs 🎛️ Configurable vision token budgets ⚡️ MTP for blazing-fast inference 💻 Optimized to run on 16GB Macs! Download it today:

Google Gemma

122,800 görüntüleme • 1 ay önce

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.
0:23

Crazy prefill speed of DwarfStar running DeepSeek v4 PRO in the DGX Station (private branch for now) using techniques that leverage the peculiar RAM + VRAM mixed setup of the Station. 3000 t/s but even more can be reached.

antirez

63,713 görüntüleme • 2 ay önce

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇
0:20

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

SGLang

99,455 görüntüleme • 1 ay önce

i made a computer vision system to monitor the situation - live webcam feed and roboflow RF-DETR for object detection - inputs are fed into SmolVLM which describes what it sees in one sentence - dataviz with vanilla js all running locally in real-time on my macbook air m2
0:27

i made a computer vision system to monitor the situation - live webcam feed and roboflow RF-DETR for object detection - inputs are fed into SmolVLM which describes what it sees in one sentence - dataviz with vanilla js all running locally in real-time on my macbook air m2

AA

72,731 görüntüleme • 8 ay önce