Skip navigation
397 BILLION PARAMETER MODEL RUNS LOCALLY ON A $32,000 DESKTOP CLUSTER
0:00 / 1:16

397 BILLION PARAMETER MODEL RUNS LOCALLY ON A $32,000 DESKTOP CLUSTER

0xMarioNawfal

@RoundtableSpace258K followers

Visit on X
144 likesDownloadOriginal post

54,183 views

397 BILLION PARAMETER MODEL RUNS LOCALLY ON A $32,000 DESKTOP CLUSTER An 8-node cluster of Nvidia GB10 mini-supercomputers pooled 1TB of memory to run Qwen 3.5 offline. Thanks to Mixture-of-Experts sparsity (activating only 17B parameters per token) the $32,000 setup delivers private 24 token/sec inference without data leaving the room.

Comments

Comments from the original post on X (Twitter)

No comments yet