The first natively trained 1-bit model: BitNet 2B. Trained... on 4 trillion tokens. that can run on CPUs like Apple M2 Native 1.58-bit weights and 8bit activations W158A8 Outperforms LLaMA &close to Qwen 2.5 1.5B in while using only 0.4GB memory versus 2GB and processes tokens 40%show more

4/ to achieve maximum memory efficiency, we quantize model... weights to ~4-bit, getting the language model under 20GB with room for the kv cache, perception encoder, and drafter alongside it. a dflash drafter proposes blocks of tokens the main model verifies in parallel, so it stays responsive.show more

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4... Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per secshow more

It's crazy that I can run MiMo 2.6 Flash... across my RTX 6000 GPU and my M5 laptop at 40 tokens/sec over 10 GbE 🤯 These are the native mxfp4 weights of a state-of-the-art model, on heterogeneous hardware. Supported out of the box in llama.cpp.show more

🔥Apple MLX first 6bit model is on Hugging Face!🔥... Qwen2.5-Coder-32B-Instruct-6bit! 3bit conversion and test in progress! Video 8x below on M4 Max 40GPU: - Prompt: 38 tokens, 61.731 tokens-per-sec - Generation: 1181 tokens, 16.939 tokens-per-sec - Peak memory: 25.122 GBshow more

I trained a tiny language model to write bad... tech jokes and run on a microcontroller 19M parameters 11 MB runs on a conference badge (RP2350), no internet, about 2 tokens a second it learned English from half a billion tokens of tiny kids stories, then I trained it on about 22,000 jokesshow more

DeepSeek R1 distilled to Qwen 1.5B easily runs on... my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:show more

You are not prepared for this, 250+ tokens/sec, 1B... model < 2GB memoryshow more

Qwen 3.6 models are now 2.5x times faster on... Atomic Chat with new MTP speedups. > MTP drafts several tokens ahead and verifies them in one pass. The speedup depends on the memory moved per pass. Users can run Qwen 3.6 models locally via the open-source Atomic Chat to test them!show more

Introducing our most accurate /search yet. We trained a... model to return the excerpts that best answer your query, giving agents highly relevant context from each result. It's SOTA on SimpleQA while using 10x fewer tokens than processing full pages. Now on by default for free.show more

One week since we dropped the weights for Beam.... 10 trillion tokens served to date. Try the model for free:show more

Ornith-1.0 35B - MLX Device - 128gb m5 max... mbp Harness - PocketAI Hub 4 bit - 11,504 tokens, 95.98 tok/s, 21.20 GB peak RAM 8 bit - 13,128 tokens, 64.96 tok/s, 38.92 GB peak RAM BF16 - 12,491 tokens, 47.72 tok/s, 70.59 GB peak RAM prompt and link to model belowshow more

🧙🏼‍♂️! We are excited to announce the launch of... $MIM and the first cauldrons on Berachain’s Artio Testnet!🐻⛓️ Users can now borrow testnet $MIM using HONEY/MIM LP tokens, natively on Artio!🔮🔥 Interact with Artio directly👇🏻show more

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude... 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.show more

GEMMA 4 FINISHED 14 MINUTES FASTER THAN QWEN 3.6... DESPITE LOWER TOKENS PER SECOND BY USING 5X FEWER TOKENSshow more

LLaVA Llama-3 and Phi-3 now on MLX 🎉🚀 You... can now run inference locally on your Mac. pip install -U mlx-vlm I’m getting ~50 tokens on a M3 Max. Model cards 👇🏾show more

🚀 Igniter & Solaxy Wallet Are LIVE! 🛸 The... wait is over. You can now launch tokens on Solaxy using Igniter, and interact with the rollup like never before using the official Solaxy Wallet! 🔹 Igniter Solaxy’s native launchpad. Launch your own token directly on the rollup. Tokens that hit a 1M $SOLX market cap are automatically rooted into Neptoon DEX.show more

MLX + M3 Ultra 512GB + Qwen3-235B-A22B-8bit = 🔥... "write a beautiful p5js particles animation that reacts to mouse clicks movements" Prompt: 22 tokens, 64.471 tokens-per-sec Generation: 6197 tokens, 18.916 tokens-per-sec Peak memory: 251.077 GBshow more

`transformers` + `torchao` quantization + `torch.compile` for faster inference... speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :show more

Today Meta released "Code Llama", a large language model... fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.show more