NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model.... Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!show more

.AI at Meta's Llama 4 Scout ripping 42.27 tok... / sec on M2 Ultra Mac Studioshow more

Tested the new MacBook Pro M4 Pro vs. the... Mac mini M1 in LM Studio, running Llama 3.2 3B 4-bit MLX. Results: M1 (8-Core) ------> 25 tok/sec M4 Pro (14-Core) -> 104 tok/sec 🤯 – 4x faster!show more

MiniMax M2.5 casually responding at ~50 tok/s with MLX... (M3 Ultra). The model was released one hour ago 🥳show more

M4 Mac Mini AI Cluster Uses EXO Labs with... Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis. The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon).show more

WhisperKit-0.7.0 is out! Single file inference is several times... faster! The demo below is running distil-whisper large-v3 at 300 tok/s and transcribes 101 seconds of audio in 1 second on an M2 Ultra Mac Studio. Details 🧵 Code (MIT): Demo Audio Input: Demo App: (TestFlight update under review)show more

Run Llama 3.1 1B offline at 60+ tok/s. Entirely... on your phone. Siri can’t save you offline, but a local model just might.show more

Making a bop folder on tik tok dm if... interested (『S』 will be included)show more

llama 3 70b beamed to my phone from my... M1 Max ~7.6 tok/s with mlx. your own little gpt-4 at homeshow more

Running DeepSeek R1 on my desk Uses EXO Labs... with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.show more

M4 Mac Mini AI cluster running Llama 3.3 70B... Preliminary results show Gigabit ethernet switch (4 tok/sec) is slower than Thunderbolt-5 with EXO Labs Benchmarks including Thunderbolt-5 coming soonTM.show more

EXO 1.0 at Apple NeurIPS booth. 4 x 512GB... M3 Ultra Mac Studios clustered all-to-all with RDMA over Thunderbolt 5. Runs DeepSeek v3.2 (8-bit) at 25 tok/sec with tensor parallelism.show more

SmallThinker 3B is one of the AI models I’d... want backed up on my phone. Here’s it is reasoning at 12 tok/s on my phone. If you’re ever stuck offline, you’ll want a model that can think.show more

Running Kimi K2.5 on my desk. Runs at 24... tok/sec with 2 x 512GB M3 Ultra Mac Studios connected with Thunderbolt 5 (RDMA) using EXO Labs / MLX backend. Yes, it can run clawdbot.show more

New Dirk Tik Tok dropped today

Qwen QwQ 32B fp16 on M4 Max and M2... Ultra powered by MLX! M2 Ultra - 10.2 toks/s M4 Max - 7.6 toks/s! "Create an amazing animation using p5js" o1-mini level local model! Note: use temp 0.7-0.75 for optimal results in coding. I did some tests and thisshow more

Running the full GLM 4.7 (8-bit) on 2 x... 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallelshow more

tik tok tik tok tik tok

NEW TIK TOK POST xxx.poppyyyy on tik tok

Hailey made a tik tok with her friends and... then Selena make the same tik tokshow more