Wide Expert Parallelism increases the total memory bandwidth available... per MoE deployment. This means the model distributes the MoE expert weights across multiple GPUs, so each GPU only needs to load a tiny fraction of the weights. This translates to higher throughput per GPU, increasing perf per dollar and perf per watt.show more

GLiNER2.5-Decide (DeBERTa-v3-large, 340M) converted to LiteRT: fp16 weights, answers... identical to the official fp32 model on 361/361 test requests, 66–70 ms per request on a Galaxy S26 GPU at 128 tokens, 175 ms at 256. Card, Python and Android sample:show more

Transformer and Mixture of Experts, explained visually! Mixture of... Experts (MoE) is a popular architecture that uses different experts to improve Transformer models. Transformer and MoE differ in the decoder block: - Transformer uses a feed-forward network. - MoE uses experts, which are feed-forward networks but smaller compared to those Transformer. During inference, a subset of experts are selected. This makes inference faster in MoE. Also, since the network has multiple decoder layers: - The text passes through different experts across layers. - The chosen experts also differ between tokens. But how does the model decide which experts should be ideal? The router does that. It is a multi-class classifier that produces softmax scores over experts to select the top K experts. The router is trained with the network, and it learns to select the best experts. But it isn't straightforward. There are challenges! Challenge 1) Notice this pattern at the start of training: - Say, the model selects "Expert 2" - This expert gets a bit better - It may get selected again since it's the "best" - It learns more - It gets selected again in the next iteration - It learns more, and so on! This means many experts can go under-trained due to the overselection of a few experts! We solve this in two steps: - Add noise to the feed-forward output of the router so that other experts can get higher logits. - Set all but the top K logits to -infinity. After softmax, these scores become zero. This way, other experts also get the opportunity to train. Challenge 2) Some experts may get exposed to more tokens than others, leading to under-trained experts. We prevent this by limiting the number of tokens an expert can process. If an expert reaches the limit, the token is passed to the next best expert. Overall, MoEs have more parameters to load. But a fraction of them are activated during inference. This leads to faster inference. Mixtral 8x7B and Llama 4 are two popular MoE-based LLMs. Have you used MoEs in production yet? To dive deeper into how MoE inference works in production, we wrote a full article covering token dispatch, grouped expert computation, model-weight memory, multi-GPU communication, expert placement, load imbalance, and performance diagnosis. Read it below.show more

MTP speedup Qwen by 2.5x in Atomic Chat Dense... vs MoE models on 2x RTX 5090 Qwen3.6 27B: 51 → 117 tps +137% Qwen3.6 35B-A3B: 218 → 267 tps +25% MTP drafts several tokens ahead and verifies them in one pass. The speedup depends on memory moved per pass. Dense 27B reads all 27B params per token, MoE 35B-A3B only reads 3B active. Dense had way more to save by batching. The baseline tps also differ (218 vs 51) for the same reason from the other side. Token generation is memory-bandwidth bound, and MoE moves ~8x less memory per token, so its baseline is already 4x ahead. ~80% draft acceptance. Zero accuracy loss. ~1 GB extra VRAM. Open-source code and local AI app – in the comments 👇show more

10 days left for GPU World, the story competition... from Neal Stephenson, Gwern, and me. What happens when frontier per-GPU capabilities diffuse?show more

GLM 5.2 INPUT FELL FROM $1.40 TO SEVEN CENTS... PER MILLION IN NINETY DAYS • what it costs now > $0.07 per million input at the cheapest of 20 providers, a 95% drop in three months. > direct still lists $1.40 in and $4.40 out, with cached input at $0.26. > Same model, same weights, twenty-fold spread depending on the door you walk through. • the free way in > New accounts on get 20M tokens: > 744B MoE, 1M context, MIT weights you can also just download and self-host. Nobody announced this. It happened one provider at a time -> and the model itself never changed. Check which provider you are actually routed through before you top up anywhere ↓show more

Native time-tracking in Notion This setup lets you track... multiple sessions per task, and shows the total time you've worked across all of them.show more

This is what happens when 48GB of VRAM isn't... enough. Someone put 4× NVIDIA RTX A6000 inside an HP Z8 Fury G5. That gives you 192GB of VRAM across four GPUs. But here's the interesting part: You don't magically get one 192GB GPU. The memory is split across four cards, and the software has to manage how the model is distributed between them. That's why these machines are interesting for local AI. The problem isn't always compute. Sometimes you just need somewhere to put the model. Would you build this today, or go for a newer GPU with more bandwidth?show more

AN AWS ENGINEER QUIETLY BUILT A 2 PETABYTE HOME... SERVER FOR $9/MONTH THAT KILLS A $3,400/MONTH CLOUD STORAGE BILL the lenovo thinkstation pgx ships nvidia's gb10 grace blackwell superchip and 128gb of unified memory in a box the size of a mac mini at 1.2kg it runs an 80b qwen3 coder model at 25 to 40 tokens per second and a 196b step-3.5-flash moe model at 20 tokens per second locally the gb10 packs 6,144 cuda cores, 192 fifth-generation tensor cores and rates at 1 petaflop of fp4 with sparsity from a single 240 watt usb-c power supply fine tuning qwen 2.5 7b with lora took 18 minutes and 41gb of unified memory while the gpu pulled 65 watts and peaked at 77 degrees the box pulls a docker container from nvidia's registry and serves a frontier model on your local network with tool calling and zero data leaving your desk bookmark this and read the article belowshow more

Only one NFL weekend since the start of the... 2010 #NFL season has recorded a higher penalty per game average than what we've seen in Week 4, 2026 That was Week 2, 2015 , averaging 18.6 penalties per game We currently have an avg of 17 penalties per game this week, so unless #ATLvsNO on MNF has 40+ penalties to increase the overall avg, the 2015 record will remain at the top of this listshow more

MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting Contributions:... • MoE-GS: the first dynamic Gaussian splatting framework employing a Mixture-of-Experts architecture, enabling robust and adaptive reconstruction across diverse dynamic scenes. • A novel Volume-aware Pixel Router integrates expert outputs through differentiable weight splatting, achieving spatially and temporally coherent adaptive blending. • Efficiency of MoE-GS is improved through single-pass multi-expert rendering and gate-aware Gaussian pruning. A separate knowledge distillation strategy trains individual experts with pseudo-labels from the MoE model, enhancing quality without modifying the architecture.show more

I’ll be telling people all 2026 and all 2027:... 2 sets per exercise per session is the answer MOST of the time 3 sets per exercise per session is the answer SOME of the time 1 sets per exercise per session is the answer OCCASIONALLY 4+ sets per exercise per session is NEVER the answer Video: Eric Janickishow more

leeknow acting cute in front of their juniors as... per han's request (again) 😭 🐰 enjoy your drink, moe moe kkyu~show more

this is the worst local AI will ever be.... tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.show more

Newborns are so tiny. Most full term infants weigh... only 6-8 pounds at birth. But cherish (and document) these special moments while they last, because you won’t have a little baby for long. In fact, children’s growth and development in the early years is positively exponential. Over the course of a single year, infants transform from tiny and helpless to increasingly verbal, mobile toddlers, typically tripling their birth weights. So quickly is your baby growing, in fact, that between their second and sixth months many increase their height by up to a quarter inch PER WEEK (1 inch per month). You’ll only have a tiny baby for the blink of an eye. So make each moment count. This happy little guy (1 week) was shared to TT by tai.vieira.21.show more

Per Nvidia CEO the next multi-trillion dollar industry is... Ai Agents. $SEN | Sentio is the biggest disruptor this cycle powered by $GPU 🧠 The ecosystem is running on every major chain. NFA, bet more on the most advanced #AI #Agent deployer ecosystem ✍🏻show more

GPU World: A story competition from Neal Stephenson, Gwern,... and me. The AI future is unevenly distributed. What would the world look like with AI broadly diffused, such as if we had one GPU per person? 1000-5000 words, fiction or non-fiction. $100K in total prizes.show more