PaliGemma 2 running 100% local, on-device, powered by MLX 🔥

Apple wasn’t kidding, the iPhone 17 Pro is really... built for running LLMs Here’s LFM2 8B A1B by running on-device with MLX in Locally AI - Local AI Chat, the iPhone runs the 8B model with zero struggle Thanks Prince Canuma for the port to MLX, it made the MLX Swift port possibleshow more

Introducing MLX-Swift-TS An SDK for running time series foundation... models fully on-device on Apple Silicon. When I joined Datadog, Inc. , I was introduced to Toto, our time series foundation model, and got excited about zero-shot forecasting across different domains. While building a health copilot app, I realized there wasn’t a simple way to run models like these locally on device. So I built one. MLX-Swift-TS exposes a common TimeSeriesForecaster interface for loading and running multiple time series architectures directly in Swift using MLX. No server required. The attached video shows on-device forecasting running inside a native Swift app. Huge thanks to Awni Hannun and the MLX team for building MLX and its Swift API, Prince Canuma for inspiration on MLX SDK patterns, and Ameet Talwalkar and the Datadog team for Toto.show more

Another demo of the iPhone 17 Pro’s on-device LLM... performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quantsshow more

GLM-4.7 Flash by Z.ai running on M3 Ultra using... MLX! 4bit at 81 toks/s 🔥 8bit at 64 toks/s 🔥 (video below) Model converted from transformers to MLX on the fly with Codex + GPT-5.2 HIgh!show more

Sam 3 by Facebook now on MLX 🚀 Here... is a realtime object tracking running on M3 Max 96GB.show more

MLX news: MOSS-TTS-Local Transformer 1.5 available on mlx-audio now!... Thanks to Prince Canuma and Lucas Newman Audio on! 🔉show more

Google’s Gemma 4 E2B running on-device on iPhone 17... Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Siliconshow more

Google's Gemma 4 26B A4B running on MacBook Pro... M5 Max 100% native Swift Achieving over 100tk/s with Apple MLX frameworkshow more

Quick demo of LFM 2.5 VL 1.6B model by... Liquid AI that I recently added to Locally AI - Local AI Chat Running locally on iPhone 17 Pro at ~90tk/s with MLX Small vision-language models are improving fastshow more

NariLabs Dia-1.6B by Toby Kim now on MLX 🚀... Get started: > pip install -U mlx-audio Model Card 👇🏾show more

1 week, 4 open-source medical AI shipments: → 35... PII models for Portuguese → openmed==1.1.0 (Brazilian + EU coverage) → OpenMedKit on iPhone: GLiNER + MLX → OpenAI's privacy-filter ported to MLX (24-33x faster) All Apache 2.0. All on-device. 15 seconds recap:show more

Qwen QwQ 32B fp16 on M4 Max and M2... Ultra powered by MLX! M2 Ultra - 10.2 toks/s M4 Max - 7.6 toks/s! "Create an amazing animation using p5js" o1-mini level local model! Note: use temp 0.7-0.75 for optimal results in coding. I did some tests and thisshow more

Llama 3.2 1B in 4-bit runs at ~60 toks/sec... with MLX Swift on my iPhone 15 pro. It's quite good and easily runs on-device:show more

Announcing MLX native Mac support for Moondream 3. Moondream... running on Mac, Linux, Windows (free!): pip install moondream-station Details:show more

DeepSeek R1 distilled to Qwen 1.5B easily runs on... my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:show more

Wow! This is amazing! Segmented every car locally in... real time with Meta's SAM3 converted to MLX. Just on-device (M2 laptop) vision getting absurdly good. Local AI is moving faster than most people realize! What other models should we test? what kind of videos?show more

10x speedup: OpenAI's privacy-filter on MLX vs CPU, side... by side in your browser. 1,818 tok/s vs 179 tok/s. Real-time PII redaction across English, German, French. On-device. No cloud. Privacy-first. Which model should we port next?show more

Running the full GLM 4.7 (8-bit) on 2 x... 512GB M3 Ultra Mac Studios Runs at 19.8 tok/sec with EXO Labs MLX RDMA backend (h/t Awni Hannun) & tensor parallelshow more

QVAC Health new beta supporting local on device model... to analyze food images, calculate calories and nutrients etc 100% privacy. Getting ready for public beta.show more