First steps for a specialized DeepSeek v4 Flash inference... engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.show more

DeepSeek DeepSeek is now live on Heurist. Open source... AI just reached new heights. Access DeepSeek-R1 and DeepSeek-V3 for smarter inference at a fraction of cost. Try it via Heurist API or explore on Pondera.show more

Got DeepSeek V4 Flash running on 2x H200s at... 160–200 tok/s on JarvisLabsAI this speed is perfect for in the loop things I do with these agents. It feels like a GLM-5.2-class model with much lower hardware requirements. I’m going to daily-drive it for a bit and see how it performs, but first impressions are really good.show more

this is the best trick to maximum usage limits... on chatgpt codex codex's best kept secret is that your main agent doesn't have to do everything... custom agents are just files in ~/.codex/agents, and one file gives you a second worker on deepseek v4 flash > create ~/.codex/agents/deepseek-worker.toml > set model = "opencode-go/deepseek-v4-flash" with model_reasoning_effort = "max" > keep it bounded: one task packet, no scope creep, report back ```toml name = "deepseek_worker" description = "bounded implementation, testing, and cleanup on deepseek v4 flash" model = "opencode-go/deepseek-v4-flash" model_reasoning_effort = "max" ``` then @ deepseek_worker in the composer... your root agent plans while the worker ships the implementation planning on the main model, execution on the flash lane... that's the whole trick (we run this exact file, last i checked it keeps the heavy turns off the main thread)show more

deepseek v4 pro is basically free right now 😳... teamorouter is offering deepseek v4 pro at almost no cost you can also use deepseek v4 flash for free what you get: - deepseek v4 pro free - deepseek v4 flash free - 1M context - openai compatible api - no subscription - credits never expire why this is worth checking: > v4 pro is currently listed at $0 input and output > flash is also available at $0 > works with claude code, codex and other tools > one api gives you access to multiple models getting started: 1. go to 2. create your account 3. open the dashboard 4. create your api key base url: 5. add it to your ai coding tool 6. select deepseek-v4-pro-free worth testing while it’s availableshow more

Running DeepSeek R1 on my desk Uses EXO Labs... with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.show more

AGI at home Running DeepSeek R1 across my 7... M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)show more

free access to 8+ top AI models 😳 APInex... is offering free models with 1M context - Gemini 3.8 Flash - Muse Spark 1.3 - GLM 5.3 Flash - Gemini 3.1 Pro - DeepSeek V4 Pro - GPT-5.6 Luna - Qwen 3.8 MAX - DeepSeek V4 Flash getting started: 1. go to 2. sign up with Gmail 3. create your API key 4. base url 5. choose a free model and start using free models, one API setup 👀show more

Batching for vision models is now available in Beta... with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mode, choose the beta runtime channel, and select LM Studio MLX v1.8.1.show more

Watching llama.cpp do 40 tok/s inference of the 7B... model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.show more

DeepSeek v4 Flash has 43 blocks... so, even 4096... batches during the prefill can be smoothly visualized. Look at the new progress bar.show more

How much faster is the new MacBook Pro for... AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.show more

QWEN 3.8 MAX + DEEPSEEK V4 PRO BOTH FREE... ON ONE ENDPOINT • the gateway > Qwen3.8-Max (2.4T MoE) + DeepSeek V4 Pro free tiers: > Sign up -> one API key -> start with no card for free tier. > Free is tight by design -- they’re letting you taste the real thing before you scale. • wiring it up > Base url: >Model IDs exact: qwen/qwen3.8-max-free and deepseek/deepseek-v4-pro-0813-free. > Drop straight into Cursor/Claude Code/any OpenAI-compatible client. Good for long agent runs you leave open on a second screen, slow for anything interactive. Point it at side projects and experiments, not at the code you would mind a stranger reading ↓show more

Another demo of the iPhone 17 Pro’s on-device LLM... performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quantsshow more

Got continuous batching working with SSMs in mlx-lm. Here's... four OpenCode agents simultaneously running Nvidia's Nemotron Nano on 64GB M4 Max. This is a nice model for smaller machines since it's MoE + hybrid attention (small cache).show more

DeepSeek V4 Flash 0731 is now 90% off on... Nous Portal for the next 7 days, in partnership with Novita AI. At this discounted price, it is over 1000x cheaper than Fable 5 on comparable tasks while still beating it on Terminal-Bench 2.1. Try it today atshow more

Today, we're excited to announce a partnership with Manta... Network (🔱,🔱), the Modular L2 solution that is transforming the landscape of ZK. Nesa is bringing private AI inference to the Manta ecosystem through a specialized collaboration. For the first time, developers on Manta can access Nesa's full library of AI models on-chain, and enjoy lightning fast, end-to-end private AI inference without ever leaving the Manta ecosystem. This means that dapps and protocols can now fuse with AI via smart contract on Manta. This integration is set to redefine the future of decentralized technology with AI. Stay tuned for more updates on how Nesa and Manta Network will be shaping the future of crypto together.show more

My AI broke the world record on Tempest yesterday!... But I still hold the human record :-) [on Extreme difficulty settings] Here's a little demo reel of the Tempest AI doing inference and training at the same time up on the hardest Tempest levels. This is all running on our Dell Technologies 7875 Workstation, with the 9995WX CPU handling 2000 fps of Tempest while the dual Blackwell RTX6000 GPUs do inference and training.show more

⭐The Year of Inference is here. Featherless is now... an official inference provider on Hugging Face, unlocking 6,700+ LLMs for anyone to run, eval, and deploy instantly. It all starts with accessibility. From DeepSeek to Mistral, LLaMA to Qwen — powerful LLMs are one click away. We believe the future of AI is shaped by the long tail: personalized, specialized models tuned to real people’s needs. To get there, inference must be open, affordable, and usable by all. Whether you're fine-tuning, prototyping, or scaling a product, this moment is for you. 🫱🏻‍🫲🏻Let’s make inference the easiest part of building with AI. 📢 Share this so more builders know what’s now possible. Excited to be partnering with clem 🤗 Julien Chaumond Vaibhav (VB) Srivastav Simon Brandeis & Hugging Face team to take this to the next level!show more

NVIDIA Nemotron 3 Nano Omni, a new multimodal reasoning... model, is now live on Jetson AI Lab and unifies vision, audio, and language into a single reasoning loop. 🙌 Power your NemoClaws by running this model with Ollama, vLLM and other inference frameworks on NVIDIA Jetson hardware. Try it ➡️show more