Qwen3-Coder 30B running locally in the @GitHub Copilot app... (ollama) spitting ~85 tokens per second on 96GB VRAM. It's not Sol, but we're getting somewhere.show more

Did I tell you that I vibe coded a... Tamagotchi style TUI for my Moltbot running on a 1400 EUR beelink AMD Ryzen AI 9 HX 370 with 64GB LPDDR5X and Ollama serving a GLM-4.7-Flash 30B at 20 tokens per secondshow more

GEMMA 4 26B ON AN RTX 4060 WITH A... 248K TOKEN CONTEXT WINDOW 20 tokens per second and a context window so large you can feed it entire codebases, books and research papers in a single prompt this is not a cloud api and not a server rack, this is a regular consumer gpu running locally with llama.cpp and q4_k_xl quantization 248k context on an 8gb vram card was not supposed to be possible and here it is just running on someone’s desk the article below covers exactly which tools and configs make this kind of setup work in 2026 ↓show more

It's over. OpenAI just crushed it. We have their... o3-level open-source model running on GroqInc at 500 tokens per second. Watch it build an entire SaaS app in just a few seconds. This is the new standard. Why the hell would you use anything else??show more

Right now, you may not have access to models... like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.show more

We are thrilled to be a launch partner for... Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯show more

A GPT-4 level chatbot, available to use completely free,... running at over 800 tokens per second on Groq. I'm genuinely mindblown by LlaMA 3. Try it with the link in the next tweet.show more

Same style prompt in Minimax H3, an AI video... model running locally on my PC at only 720p resolution, 15-second generation. I took some "liberties" with the prompt to test some physics... Now, compared to Seedance 2.5 or many of the top closed models, Minimax H3 running locally, quantized and optimized for speed, is definitely weaker at following prompts, but it's just amazing that we have an open-weight model that's this powerful and not some closed-source cloud model. It missed several things, like the barking, the cart turning into a bicycle, and she drops the bags at the beginning, along with some ghosting issues, but it's pretty darn good. Also... did I mention it's uncensored?show more

AN AWS ENGINEER QUIETLY BUILT A 2 PETABYTE HOME... SERVER FOR $9/MONTH THAT KILLS A $3,400/MONTH CLOUD STORAGE BILL the lenovo thinkstation pgx ships nvidia's gb10 grace blackwell superchip and 128gb of unified memory in a box the size of a mac mini at 1.2kg it runs an 80b qwen3 coder model at 25 to 40 tokens per second and a 196b step-3.5-flash moe model at 20 tokens per second locally the gb10 packs 6,144 cuda cores, 192 fifth-generation tensor cores and rates at 1 petaflop of fp4 with sparsity from a single 240 watt usb-c power supply fine tuning qwen 2.5 7b with lora took 18 minutes and 41gb of unified memory while the gpu pulled 65 watts and peaked at 77 degrees the box pulls a docker container from nvidia's registry and serves a frontier model on your local network with tool calling and zero data leaving your desk bookmark this and read the article belowshow more

They are not online or poor network? #Diaspora Call... 🇺🇿 Nigeria at $0.00167 per second. Stop paying for minute you didn’t use. Make the switch now & save 90% on your calls to Nigeria Get it App now! Living in the diaspora, you know how it goes—you need to check in on a project or family back home, but the person on the other end is offline, or the data connection just isn't holding up. SoftTalk Messenger is the solution An African super calling app - charges by per second — not per minute. • ✅ Rate $0.00167 / secshow more

Aaron Boone was asked if he thinks the Yankees... are "running out of time" to get themselves on track against teams above .500: "No, we're not running out of time, but if we don't do better, then it's going to fizzle out and we're not going to get to where we want to be."show more

China f*cked the world again DeepSeek just dropped a... killer library that saves up to 98% on tokens ur GPT Astra and Opus 5.5 it's DeepGEMM, the engine DeepSeek trains and runs its own models on faster GPU math -> more tokens per GPU -> cheaper tokens > plug the engine DeepSeek uses into your own models > squeeze more tokens per second out of the GPUs you already pay for > match or beat expert-tuned GPU libraries on speed u will pay $1.6 instead of $50 its sitting in one public repo, link belowshow more

Fable 5 absolutely crushed the HTML5 physics contest, but... cost 6x more than Opus 4.8 and 39× more than GLM 5.2 in that test. Test was done on atomic[.]chat, a desktop app that runs LLMs locally. The test asked 4 models to generate self-contained canvas demos with believable motion and collisions. The scenes were not simple animations because every crash needed gravity, force, timing, and contact handling. Outputs: - Fable 5: 62,158 tokens, $3.12 - GPT 5.5: 37,753 tokens, $1.14 - Opus 4.8: 22,280 tokens, $0.56 - GLM 5.2: 36,246 tokens, $0.08show more

GROK CODE IS WIPING THE FLOOR WITH EVERYONE It... launched 5 months ago and it's already the most-used coding model out there. On the Kilo Code leaderboard, it's getting used twice as much as the second-place model in almost every category. In 2025, it’s way ahead on OpenRouter too, with over 16 trillion tokens processed. Nothing else is even close. Source: X Freezeshow more

This is running in my web browser on my... laptop. It's deterministic and running using webgpu Tonight; I have a /goal going where it is going to set up one of my 4090s as the authority server, and then test all the rollback code. Goal is 50k boxes simmed per user, up to 512 users. Massively multiplayer massively rigid body physics online games inside your web browser. Why the hell not, we've got the tokens to spare!show more

Jaylen Brown was doubled on 34.7% of his touches... last night, the most in any game of his career where he had 70+ touches. It worked out for OKC in the 1st half, with the Celts getting just 0.25 points per direct touch when JB was doubled. In the second half? Not so much, with BOS getting 1.7 points per direct touch when JB saw twoshow more

50% more context unlocked for Qwen 3.8 27b Q4_K_XL... dflash 2 on a single RTX 4090 (24 GB VRAM) I found a hidden VRAM tax in llama.cpp. By combining my custom 2 bit DFlash 2 drafter with one overlooked server flag, I just unlocked another +80,000 tokens of context. Qwen3.8-27B is now running a massive 250,000 context at 75 tokens/s on a single RTX 4090. Here is the secret: By default, `llama-server` reserves massive chunks of your VRAM to handle multiple concurrent users (batching). If you are running a single user session, you are bleeding memory for features you aren't using. By passing the `--parallel 1` flag, you force the engine to dedicate 100% of your 24GB VRAM buffer to a single user. When we combine the VRAM saved by our Q2_K 2-bit drafter with the VRAM saved by `--parallel 1`, the context ceilings absolutely explode: Note: all benchmarks carried out with a massive 28k prompt. Ubuntu 22. ### THE NEW 24GB PHYSICAL LIMITS (Single RTX 4090): # 1. The "Repo Swallower" (Q4 KV Cache): - Context: 250,000 tokens (Up from 170k!) - Speed: 73.66 t/s decode | 1,608 t/s prefill - Peak VRAM: 23.8 GB # 2. The "High-Precision SWE" (Q8 KV Cache): - Context: 150,000 tokens (Up from 100k!) - Speed: 75.01 t/s decode | 1,667 t/s prefill - Peak VRAM: 23.9 GB # 3. The "Pristine Attention" (Unquantized FP16 KV): - Context: 90,000 tokens - Speed: 80.58 t/s decode | 1,699 t/s prefill - Peak VRAM: 23.92 GB ### HOW TO RUN THE 250K GOD STACK TODAY: (Requires PR #27342 + my Q2_K Hugging Face drafter) llama.cpp flags: ./build/bin/llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -md Qwen3.8-27B-DFlash2-Q2_K.gguf --spec-type draft-dflash --spec-draft-n-max 3 -c 250000 -ngl 99 --parallel 1 --port 8080 -ctv q4_0 -ctk q4_0 We are pushing a quarter million tokens of context with speculative DFlash 2 decoding at 73 tokens/second on a single consumer gaming GPU. I dropped my custom 2 bit Hugging Face GGUF links, visual performance graphs, and the PR #27342 build instructions in the replies below. If you own a single RTX 3090 or 4090, it is officially time to cancel your API subscriptions and let local silicon eat the cloud. how much monthly API spend does an optimized 4090 rig like this actually replace for you?show more

Sightings have been reported of a Chaos Emerald making... landfall somewhere in San Diego on Saturday July 15th. We're almost there but the job's not done… Be ready to secure each and every Emerald so Sonic can power Club Chaos! Join the Chaos Hunt:show more

Been using Qwen 3.8 27B (Q4) locally on 64GB... of VRAM. Here is the verdict: SLOW 18 tps with ZERO system prompt to process and that degrades significantly with a harness system prompt and as the context window grows. RIP if you have to compact. I had it implement this PRD and it's been running for 6 hours. By comparison Grok 4.6 and Kimi K3 hosted finished in about ~30 minutes. High hopes, but these 27B variants are too dense. This is not a consumer grade local model - and I consider consumer grade to be anything up to $5000.show more

The best open source 3D AI generation model now... runs natively in ComfyUI. The workflow runs 100% locally, on just 6GB of VRAM. No more complex setup or dependency errors. TRELLIS.2 and Pixal3D work right out of the box, with one click. Not to mention, it's FULLY COMMERCIAL-SAFE. The team re-implemented and removed every strictly-licensed library (thanks to the legend Kijai). Drop in an image and click run, it's that easy to start. Go play with it. Free workflow below!show more