正在加载视频...
视频加载失败
> be Kimi K3 > drop July 16 > 2.8 trillion parameters > the biggest open model ever released > 1 million token context. native multimodal. > Kimi Delta Attention — 6.3x faster decoding > priced at a third of Fable 5 > people rush in so fast the... show more
0 条评论
暂无评论
原始帖子的评论将显示在这里
![Its not every day you wake up to find that the Pope has made your lifes work the central focus of his papacy: “Disarming AI means freeing it from the mentality of “armed” competition [..] This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance." - POPE LEO XIV, May 2026 Here, “disarmed” means “neutralised” in the sense that this should not be a differentiator The Innovation Game (TIG) was created to keep data and algorithms open, in order to prevent monopolistic control It's not just an aspiration, it's an economic mechanism that makes open data and open algorithms the rational economic choice • All algorithms are published openly by TIG • If you are willing to make the data you process with an algorithm open, you can use it free-of-charge • Alternatively, if you would like to keep this data private, there is a fee to pay for using the algorithm • All fees are used to fund more open innovation Its an elegant, global, self-reinforcing engine The logical end point of monopoly is that innovation stops We cannot allow that to happen Pope Leo XIV I would be grateful for your thoughts on The Innovation Game](https://image.24vids.com/tw-2059651921036620085/amplify_video_thumb/2059646214828638208/img/uoICLB_LNotL5PZL.jpg)
![I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.](https://image.24vids.com/tw-2076665127450067116/amplify_video_thumb/2076664980515278849/img/K3GTizaUDbpZHQuo.jpg)
