Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

We also tested Ternary Bonsai 2 27B on the 2026 IMO problems against full-precision Qwen3.8 27B (54GB) and Gemma 4 12B QAT (~7GB). No internet. No tools. 131K-token reasoning budget. Bonsai scored in the upper end of the human bronze-medal range and retained 95% of Qwen’s IMO score, while... show more

PrismML

28,279 subscribers

34,109 görüntüleme • 9 gün önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

7 Yorum

PrismML profil fotoğrafı

PrismML9 gün önce

First up: building a 3D skateboard game from scratch in a single HTML file. The first attempt isn't perfect. Instead of restarting a better generation, we keep the same conversation going, point out the issues, and give the model feedback. It progressively fixes the gameplay, visuals, and controls over several iterations. That iterative loop being able to take feedback, debug its own work, and improve an existing artifact is much closer to how we think language models are most useful in practice. Code:

PrismML profil fotoğrafı

PrismML9 gün önce

We have been very excited to see how the community has been utilizing our Bonsai 2 model, well beyond use cases that we had originally envisioned it for. We’ve spent the last few days putting Bonsai 2 27B through various examples inspired by what the community has shown us. These examples showcase the strengths, and some of the shortcomings–particularly on longer multi-turn agentic workflows, and will be extremely helpful as we continue to improve. We’ll be sharing a few of those demos, along with some practical guidance on the settings and prompting that get the best results from the model. Demo repo:

PrismML profil fotoğrafı

PrismML9 gün önce

Starting from an empty workspace, Bonsai 2 27B built a browser-based desktop in a single HTML file, then iteratively fixed issues across multiple rounds of feedback, from broken windows and UI behavior to complete functionality and final styling. Code:

PrismML profil fotoğrafı

PrismML9 gün önce

We’ll keep sharing both what the model does well and where we’re working to make it better. More updates soon.

BirchEater67 profil fotoğrafı

BirchEater679 gün önce

What is going on in Gemma's CoT lol

luresh profil fotoğrafı

luresh9 gün önce

When availability on lmstudio?

Craig Van profil fotoğrafı

Craig Van9 gün önce

Are you gonna do one of the MoE versions anytime?

Benzer Videolar

Try Ternary Bonsai 27B directly on Hugging Face , powered by Together AI

Try Ternary Bonsai 27B directly on Hugging Face , powered by Together AI

PrismML

19,934 görüntüleme • 2 ay önce

Here is Ternary Bonsai 27B running an end-to-end agentic workflow locally with Hermes on an NVIDIA GeForce RTX 5090 GPU. The model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files, intermediate states, and iterations remain local.

Here is Ternary Bonsai 27B running an end-to-end agentic workflow locally with Hermes on an NVIDIA GeForce RTX 5090 GPU. The model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files, intermediate states, and iterations remain local.

PrismML

154,473 görüntüleme • 2 ay önce

NEW: Ternary Bonsai 2 just landed on Hugging Face — a 27B reasoning model in ternary weights, built for the agentic era 🤯 9x smaller than FP16 while retaining 98.2% of the intelligence. At <6GB in size, it can even run 100% locally in your browser on WebGPU! Try it yourself 👇

NEW: Ternary Bonsai 2 just landed on Hugging Face — a 27B reasoning model in ternary weights, built for the agentic era 🤯 9x smaller than FP16 while retaining 98.2% of the intelligence. At <6GB in size, it can even run 100% locally in your browser on WebGPU! Try it yourself 👇

Xenova

17,717 görüntüleme • 16 gün önce

The phone threshold is even harder than the storage number suggests. A phone exposes only part of its memory to an application, and the model must share that budget with its KV cache, activations, runtime, and the rest of the product. At 3.9 GB, 1-bit Bonsai 27B clears that threshold with room to work, making 27B-class local AI possible on a phone for the first time. (Demo Mode: cached & prefilled image context)

The phone threshold is even harder than the storage number suggests. A phone exposes only part of its memory to an application, and the model must share that budget with its KV cache, activations, runtime, and the rest of the product. At 3.9 GB, 1-bit Bonsai 27B clears that threshold with room to work, making 27B-class local AI possible on a phone for the first time. (Demo Mode: cached & prefilled image context)

PrismML

33,665 görüntüleme • 2 ay önce

A 27B parameter model used to need a server room. Now it runs on: • iPhone • Android • Mac PrismML's Bonsai makes it possible. • 1-bit weights • 27B params in just 3.9GB • ~90% of full precision quality (PrismML evals) • Better than the 2-bit version at less than half the size The entire Bonsai family is now live in RunAnywhere. • 1.7B to 27B models • 1-bit + 2-bit ternary • Thinking mode On iOS and Mac, Bonsai runs through ggml's llama.cpp and Apple's MLX. On Android, Bonsai runs through ggml's llama.cpp and directly on the Qualcomm Hexagon NPU through QHexRT, our proprietary inference runtime. We built custom silicon kernels to make 1-bit inference on an NPU possible for the first time. Two years ago this needed a data center. Now it thinks in airplane mode. Now available in the RunAnywhere app on the App Store and Google Play.

A 27B parameter model used to need a server room. Now it runs on: • iPhone • Android • Mac PrismML's Bonsai makes it possible. • 1-bit weights • 27B params in just 3.9GB • ~90% of full precision quality (PrismML evals) • Better than the 2-bit version at less than half the size The entire Bonsai family is now live in RunAnywhere. • 1.7B to 27B models • 1-bit + 2-bit ternary • Thinking mode On iOS and Mac, Bonsai runs through ggml's llama.cpp and Apple's MLX. On Android, Bonsai runs through ggml's llama.cpp and directly on the Qualcomm Hexagon NPU through QHexRT, our proprietary inference runtime. We built custom silicon kernels to make 1-bit inference on an NPU possible for the first time. Two years ago this needed a data center. Now it thinks in airplane mode. Now available in the RunAnywhere app on the App Store and Google Play.

RunAnywhere (YC W26)

17,587 görüntüleme • 2 ay önce

Darkbloom is the first model provider to support Ternary Bonsai 2 27B -- concentrated intelligence that fits on your phone. Try it now: First 250 users get 100 million free tokens; PrismML's new flagship model: - a ternary compression of Qwen3.8 27B at 2bits per parameter. - 8.5 GB total, 5x smaller than original - keeps 98.2% of the Qwen's FP16 benchmark performance. - 75% cheaper than Qwen 27B. Qwen3.8 27B already operates comparably with Opus 4.6 and 5.6 Luna on certain tasks. This one does it in the memory of a phone. 262K context, image input, Apache 2.0. From our first run on the network, on a single M5 Max with no caching: - 35 tok/s decode at 1K context, - 31 tok/s at 10K, - 19 tok/s at 50K. But we expect more performance gain coming in a few weeks! That's a full 27B reasoning model running comfortably on any Mac. 1,000+ Macs are serving on Darkbloom right now. Go try it out!! Thank you to Babak Hassibi Sahin Lale Omead Pooladzandi Reza Sadri Tushar Bansal Evan Walters and the whole PrismML team. This is exactly the kind of model Darkbloom was built for. You can read the essay by Bonsai on Why Local AI Matters:

Darkbloom is the first model provider to support Ternary Bonsai 2 27B -- concentrated intelligence that fits on your phone. Try it now: First 250 users get 100 million free tokens; PrismML's new flagship model: - a ternary compression of Qwen3.8 27B at 2bits per parameter. - 8.5 GB total, 5x smaller than original - keeps 98.2% of the Qwen's FP16 benchmark performance. - 75% cheaper than Qwen 27B. Qwen3.8 27B already operates comparably with Opus 4.6 and 5.6 Luna on certain tasks. This one does it in the memory of a phone. 262K context, image input, Apache 2.0. From our first run on the network, on a single M5 Max with no caching: - 35 tok/s decode at 1K context, - 31 tok/s at 10K, - 19 tok/s at 50K. But we expect more performance gain coming in a few weeks! That's a full 27B reasoning model running comfortably on any Mac. 1,000+ Macs are serving on Darkbloom right now. Go try it out!! Thank you to Babak Hassibi Sahin Lale Omead Pooladzandi Reza Sadri Tushar Bansal Evan Walters and the whole PrismML team. This is exactly the kind of model Darkbloom was built for. You can read the essay by Bonsai on Why Local AI Matters:

Gajesh

48,051 görüntüleme • 16 gün önce

$PrismML Releases Bonsai 27B: 1-bit and Ternary Builds of Qwen3.6-27B Hitting 89.5% of FP16 at 3.9GB. No new pretrain. No higher-precision escape hatches. No multi-GPU rig. Here's how it works. 👇 1: Codes, not floats Every weight becomes a code, with one shared FP16 scale per group of 128. Ternary is {−1, 0, +1}, binary is {−1, +1}. Sharing the scale across 128 weights keeps its cost at 16/128 = 0.125 bits. → Ternary: log2(3) + 16/128 ≈ 1.71 bits/weight → 5.9GB → Binary: 1 + 16/128 = 1.125 bits/weight → 3.9GB 2: Post-training, not from scratch No BitNet-style low-bit pretrain. It starts from off-the-shelf Qwen3.6-27B, architecture unchanged. The representation runs end to end across embeddings, attention projections, MLP projections, and the LM head. → 9.4× (ternary) and 14.2× (binary) vs the 54GB FP16 baseline 3: Labels are not bit-widths Conventional low-bit builds are mixed-precision by construction. The advertised name describes the most-compressed tensors, not the model. → Q4_K_XL, labeled "4-bit," is really 5.2 bits/weight at 17.6GB → IQ2_XXS, labeled "2-bit," is really 2.8 bits/weight at 9.4GB 4: Fitting a phone is two budgets iOS caps a single app near half of RAM, so a 12GB iPhone exposes ~6GB. The KV cache grows on top. Hybrid attention at ~75% linear means only 16 of 64 layers cache. → 4-bit KV: 4.3GB at 262K context, down from 17.2GB → 11.0 tok/s on iPhone 17 Pro Max 5: The numbers (15 benchmarks, thinking mode) → Ternary: 80.49 avg at 5.9GB — 94.6% of FP16 → 1-bit: 76.11 avg at 3.9GB — 89.5% of FP16 → IQ2_XXS falls to 57.5 on AIME26 while still scoring 88.93 on MMLU-Redux The key takeaway: 27B-class reasoning without the 54GB checkpoint — group-wise ternary and binary codes, an end-to-end low-bit language stack, 4-bit KV, on one phone. Full analysis: Repo: Model weight: Technical details: PrismML$

PrismML Releases Bonsai 27B: 1-bit and Ternary Builds of Qwen3.6-27B Hitting 89.5% of FP16 at 3.9GB. No new pretrain. No higher-precision escape hatches. No multi-GPU rig. Here's how it works. 👇 1: Codes, not floats Every weight becomes a code, with one shared FP16 scale per group of 128. Ternary is {−1, 0, +1}, binary is {−1, +1}. Sharing the scale across 128 weights keeps its cost at 16/128 = 0.125 bits. → Ternary: log2(3) + 16/128 ≈ 1.71 bits/weight → 5.9GB → Binary: 1 + 16/128 = 1.125 bits/weight → 3.9GB 2: Post-training, not from scratch No BitNet-style low-bit pretrain. It starts from off-the-shelf Qwen3.6-27B, architecture unchanged. The representation runs end to end across embeddings, attention projections, MLP projections, and the LM head. → 9.4× (ternary) and 14.2× (binary) vs the 54GB FP16 baseline 3: Labels are not bit-widths Conventional low-bit builds are mixed-precision by construction. The advertised name describes the most-compressed tensors, not the model. → Q4_K_XL, labeled "4-bit," is really 5.2 bits/weight at 17.6GB → IQ2_XXS, labeled "2-bit," is really 2.8 bits/weight at 9.4GB 4: Fitting a phone is two budgets iOS caps a single app near half of RAM, so a 12GB iPhone exposes ~6GB. The KV cache grows on top. Hybrid attention at ~75% linear means only 16 of 64 layers cache. → 4-bit KV: 4.3GB at 262K context, down from 17.2GB → 11.0 tok/s on iPhone 17 Pro Max 5: The numbers (15 benchmarks, thinking mode) → Ternary: 80.49 avg at 5.9GB — 94.6% of FP16 → 1-bit: 76.11 avg at 3.9GB — 89.5% of FP16 → IQ2_XXS falls to 57.5 on AIME26 while still scoring 88.93 on MMLU-Redux The key takeaway: 27B-class reasoning without the 54GB checkpoint — group-wise ternary and binary codes, an end-to-end low-bit language stack, 4-bit KV, on one phone. Full analysis: Repo: Model weight: Technical details: PrismML

Marktechpost AI

31,860 görüntüleme • 2 ay önce

Bonsai 27B just changed the local LLM game forever. 1-bit quantization shrinks it from 54GB to just 3.8GB (-93%), while retaining 90% of its intelligence. That's insane. With custom WebGPU kernels written by Fable 5 and GPT 5.6 Sol, the model now runs locally in your browser!

Bonsai 27B just changed the local LLM game forever. 1-bit quantization shrinks it from 54GB to just 3.8GB (-93%), while retaining 90% of its intelligence. That's insane. With custom WebGPU kernels written by Fable 5 and GPT 5.6 Sol, the model now runs locally in your browser!

Xenova

211,255 görüntüleme • 2 ay önce

We’re also launching Bonsai Studio, an iOS app for trying Bonsai Image 4B directly on your iPhone. Run image generation on-device, with no subscription and no API requests.

We’re also launching Bonsai Studio, an iOS app for trying Bonsai Image 4B directly on your iPhone. Run image generation on-device, with no subscription and no API requests.

PrismML

14,499 görüntüleme • 4 ay önce

bonsai 2 27b on an rtx 3060 12gb, the full receipt sheet. save this one, the 12gb row of the small gpu guide is built from it. speed by depth, then what context costs, live server, thinking on, real sessions > 7k deep: 24.4 tok/s > 12k deep: 21.9 tok/s > 35k deep: 17.8 tok/s > 77k deep: 13.0 tok/s > 64k window: 7.3gb resident > 128k window: 8.8gb resident > 192k window: 10.2gb resident > 262k window: 11.7gb resident, 0.6gb to spare, the whole native window on a 12gb card > every 64k of context costs 1.47gb, so 327k would not fit > a 41,312 token build session from 35k to 77k of context averaged 15.0 tok/s across 46 minutes > prefill 295 tok/s at 2k of context, 243 tok/s at 35k, first token in 0.6 seconds, fresh decode 26.1 tok/s > the card pinned 149 of 150 w the entire time, 78c, fan at 80%, power bound, not heat bound > 0.158 tok/s per watt at the fresh end the setup > model: ternary bonsai 2 27b, PTQ1_0, 1.75 bits per weight, 5.95gb on disk, base qwen 3.8 27b, apache 2.0 > runtime: prismml llama.cpp fork, prebuilt cuda 12.4 binary, no compile > serve: full 262k native window resident, q4 kv cache, flash attention, one slot, 11.7 of 12gb in use for anyone who followed bonsai 1 in july, that was the 3.9gb 1bit file at 42 tok/s on a 3060 ti, a faster card and a smaller file, so the same card comparison is not on the table yet, it comes with the 8gb test. what changed is the base, qwen 3.8 instead of 3.6, and the retention, 98.2% on their suite instead of 95%, and the whole 262k window fitting on 12gb.

bonsai 2 27b on an rtx 3060 12gb, the full receipt sheet. save this one, the 12gb row of the small gpu guide is built from it. speed by depth, then what context costs, live server, thinking on, real sessions > 7k deep: 24.4 tok/s > 12k deep: 21.9 tok/s > 35k deep: 17.8 tok/s > 77k deep: 13.0 tok/s > 64k window: 7.3gb resident > 128k window: 8.8gb resident > 192k window: 10.2gb resident > 262k window: 11.7gb resident, 0.6gb to spare, the whole native window on a 12gb card > every 64k of context costs 1.47gb, so 327k would not fit > a 41,312 token build session from 35k to 77k of context averaged 15.0 tok/s across 46 minutes > prefill 295 tok/s at 2k of context, 243 tok/s at 35k, first token in 0.6 seconds, fresh decode 26.1 tok/s > the card pinned 149 of 150 w the entire time, 78c, fan at 80%, power bound, not heat bound > 0.158 tok/s per watt at the fresh end the setup > model: ternary bonsai 2 27b, PTQ1_0, 1.75 bits per weight, 5.95gb on disk, base qwen 3.8 27b, apache 2.0 > runtime: prismml llama.cpp fork, prebuilt cuda 12.4 binary, no compile > serve: full 262k native window resident, q4 kv cache, flash attention, one slot, 11.7 of 12gb in use for anyone who followed bonsai 1 in july, that was the 3.9gb 1bit file at 42 tok/s on a 3060 ti, a faster card and a smaller file, so the same card comparison is not on the table yet, it comes with the 8gb test. what changed is the base, qwen 3.8 instead of 3.6, and the retention, 98.2% on their suite instead of 95%, and the whole 262k window fitting on 12gb.

Sudo su

31,174 görüntüleme • 16 gün önce

watch bonsai 3.9gb model crush tool calls. this is running in hermes agent locally, and it doesn't just fire blind, it reasons first before every single action, thinks through what it actually needs, then makes the call. zero failures the whole run, not one dropped, not one malformed, across a full multi step agent loop. for anyone new here, bonsai is a 1 bit quant of qwen 3.6 27b dense, the exact model i spent all month calling the king of the 3090. PrismML took the king, crushed it down to a single sign bit per weight, 3.9gb, small enough to sit next to a gemma 12b, and it still holds an agent loop and reasons before it acts. sub 4 bit is where models are supposed to turn to mush. someone even clocked it at tool-calling 0 out of 100. not in my runs. this is the warm-up. the real test is the build gauntlet my q4 27b nailed, elements, particle swarm, mandelbrot, and openclaw runs the same one right after. we find out if 1bit actually keeps up.

watch bonsai 3.9gb model crush tool calls. this is running in hermes agent locally, and it doesn't just fire blind, it reasons first before every single action, thinks through what it actually needs, then makes the call. zero failures the whole run, not one dropped, not one malformed, across a full multi step agent loop. for anyone new here, bonsai is a 1 bit quant of qwen 3.6 27b dense, the exact model i spent all month calling the king of the 3090. PrismML took the king, crushed it down to a single sign bit per weight, 3.9gb, small enough to sit next to a gemma 12b, and it still holds an agent loop and reasons before it acts. sub 4 bit is where models are supposed to turn to mush. someone even clocked it at tool-calling 0 out of 100. not in my runs. this is the warm-up. the real test is the build gauntlet my q4 27b nailed, elements, particle swarm, mandelbrot, and openclaw runs the same one right after. we find out if 1bit actually keeps up.

Sudo su

55,466 görüntüleme • 2 ay önce

Which LLMs actually love to think? Tested 7 models on 5 math problems, measured reasoning length. The think winners: both Qwen3.5 models (27B and 35B A3B) — massive overthinkers, up to 10k+ tokens on a single question. Plot twists: > Kimi K2.6 feels verbose, actually one of the leanest > Gemma4 26B A4B solved 2 with ZERO thinking

Which LLMs actually love to think? Tested 7 models on 5 math problems, measured reasoning length. The think winners: both Qwen3.5 models (27B and 35B A3B) — massive overthinkers, up to 10k+ tokens on a single question. Plot twists: > Kimi K2.6 feels verbose, actually one of the leanest > Gemma4 26B A4B solved 2 with ZERO thinking

stevibe

97,403 görüntüleme • 5 ay önce

this is what 12gb of vram builds in 2026, absolute magic > rtx 3060 12gb, #1 gpu on steam > bonsai 2 27b + mtp, 5.95 gb of weights > hermes agent, 5 hours, 328k tokens written > 8 js files, 2,368 lines, zero hand written code > 50 tok/s fresh, 22 tok/s average, 125k context watch the full video, 5 hours in 12 minutes of pure dance of a local ai model on rtx 3060 12gb vram, and stay till the end for the full gameplay. this entire game was built by bonsai2, a qwen 3.8 27b dense compressed to ternary, and this small model is punching way above its weight. it built multi file engineering work using hermes agent, sure it's not fast but perfect for overnights and routine work and the quality is insane, and context holding is another best one, it does not lose the thread. i ran PrismML bonsai 1 made from qwen 3.6 27b dense and i built things with it, but this time with the latest base model qwen 3.8 these results are insane, and because i loved building with it a lot i thought many more of you would run it because this gpu exists in almost every home. so i packaged mtp, doubled the speed from 26 tok/s to 50 tok/s, packed the prefill fix in and released it on huggingface, almost 4,000 downloads in 3 days. i'll leave a link below.

this is what 12gb of vram builds in 2026, absolute magic > rtx 3060 12gb, #1 gpu on steam > bonsai 2 27b + mtp, 5.95 gb of weights > hermes agent, 5 hours, 328k tokens written > 8 js files, 2,368 lines, zero hand written code > 50 tok/s fresh, 22 tok/s average, 125k context watch the full video, 5 hours in 12 minutes of pure dance of a local ai model on rtx 3060 12gb vram, and stay till the end for the full gameplay. this entire game was built by bonsai2, a qwen 3.8 27b dense compressed to ternary, and this small model is punching way above its weight. it built multi file engineering work using hermes agent, sure it's not fast but perfect for overnights and routine work and the quality is insane, and context holding is another best one, it does not lose the thread. i ran PrismML bonsai 1 made from qwen 3.6 27b dense and i built things with it, but this time with the latest base model qwen 3.8 these results are insane, and because i loved building with it a lot i thought many more of you would run it because this gpu exists in almost every home. so i packaged mtp, doubled the speed from 26 tok/s to 50 tok/s, packed the prefill fix in and released it on huggingface, almost 4,000 downloads in 3 days. i'll leave a link below.

Sudo su

76,833 görüntüleme • 11 gün önce

Qwopus 3.6-27b Coder I had a lot of requests to test it, so I did. I ran the same tests I’ve done on other models. It scored lower on tool-eval-bench than both Qwen 3.6 27b and Qwen 3.6 35b. It also produced broken single-file Tetris and Solar System HTML demos, even after 3 fresh attempts. In the Tetris html, the game stalled when you completed a line. It also randomly changed the pieces when moving them left or right. In the Solar System html, clicking on the sun or any planet broke the rendering — the objects just disappeared. Conclusion: The regular Qwen 3.6 27b and 35b models performed better. What do you think of Qwopus 3.6-27b Coder? Full results & RAW files: Full prompts in the posts below.

Qwopus 3.6-27b Coder I had a lot of requests to test it, so I did. I ran the same tests I’ve done on other models. It scored lower on tool-eval-bench than both Qwen 3.6 27b and Qwen 3.6 35b. It also produced broken single-file Tetris and Solar System HTML demos, even after 3 fresh attempts. In the Tetris html, the game stalled when you completed a line. It also randomly changed the pieces when moving them left or right. In the Solar System html, clicking on the sun or any planet broke the rendering — the objects just disappeared. Conclusion: The regular Qwen 3.6 27b and 35b models performed better. What do you think of Qwopus 3.6-27b Coder? Full results & RAW files: Full prompts in the posts below.

Mia

11,183 görüntüleme • 3 ay önce

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99 flags. same 48k context. unsloth gemma-4-12b-it-Q4_K_M.gguf → 15 tok/sec @ 48k ctx unsloth gemma-4-12B-it-qat-UD-Q4_K_XL.gguf → 32 tok/sec @ 48k ctx → 26 tok/sec @ 64k ctx 64k context is a big deal. Hermes 3 agent requires 64k minimum to run. you're now getting full hermes compatible context on a budget consumer GPU at 26 tok/sec locally. 2.1x faster on identical hardware. and here's the part that breaks your brain: the QAT-UD-Q4_K_XL is actually SMALLER than the Q4_K_M "XL" why? QAT = Quantization Aware Training Google didn't train the model first and compress it later they trained it to be quantized from day one the weights already know how to survive low precision that's why you get more quality per byte llamacpp flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -cnv -ngl 99 -c 48000 -v fits in 8GB VRAM clean. no API. no cloud. no subscription. and this isn't even the MTP variant yet Gemma-4-E2B QAT runs on 3GB RAM, E4B on 5GB, 12B on 7GB, 26-A4B on 15GB and 31B on 18GB. I have benchmarked the 26b and 31b qat as well on a single RTX 4090, checkout the comments for details. If you have a 6GB or 8GB VRAM GPU, post your numbers. more benchmarks and configs coming soon

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99 flags. same 48k context. unsloth gemma-4-12b-it-Q4_K_M.gguf → 15 tok/sec @ 48k ctx unsloth gemma-4-12B-it-qat-UD-Q4_K_XL.gguf → 32 tok/sec @ 48k ctx → 26 tok/sec @ 64k ctx 64k context is a big deal. Hermes 3 agent requires 64k minimum to run. you're now getting full hermes compatible context on a budget consumer GPU at 26 tok/sec locally. 2.1x faster on identical hardware. and here's the part that breaks your brain: the QAT-UD-Q4_K_XL is actually SMALLER than the Q4_K_M "XL" why? QAT = Quantization Aware Training Google didn't train the model first and compress it later they trained it to be quantized from day one the weights already know how to survive low precision that's why you get more quality per byte llamacpp flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -cnv -ngl 99 -c 48000 -v fits in 8GB VRAM clean. no API. no cloud. no subscription. and this isn't even the MTP variant yet Gemma-4-E2B QAT runs on 3GB RAM, E4B on 5GB, 12B on 7GB, 26-A4B on 15GB and 31B on 18GB. I have benchmarked the 26b and 31b qat as well on a single RTX 4090, checkout the comments for details. If you have a 6GB or 8GB VRAM GPU, post your numbers. more benchmarks and configs coming soon

Alok

264,062 görüntüleme • 4 ay önce

1. 70% of files in Clinics belong to Illegal Immigrants from Zimbabwe 2. Schools are full of Illegal Immigrants 3.State of Emergency is needed on Illegal immigrants And 1, 2 and 3 are not topics in the National Dialogue. The Anc has designed the Agenda of the National Dialogue The Anc is 100% Cause of the Problems in our Country The Anc wanted to assist us in the Problems they have caused, they also dictate the Agenda. It will never Happen. You cannot discuss eradication of Malaria with Mosquitoes. Anc must Go #VoetsekANC

1. 70% of files in Clinics belong to Illegal Immigrants from Zimbabwe 2. Schools are full of Illegal Immigrants 3.State of Emergency is needed on Illegal immigrants And 1, 2 and 3 are not topics in the National Dialogue. The Anc has designed the Agenda of the National Dialogue The Anc is 100% Cause of the Problems in our Country The Anc wanted to assist us in the Problems they have caused, they also dictate the Agenda. It will never Happen. You cannot discuss eradication of Malaria with Mosquitoes. Anc must Go #VoetsekANC

KommanderJosh III

66,751 görüntüleme • 1 yıl önce

The greatest constraint on exercising RL at scale on real world problems is the lack of data on end goals. Also, the lack of variation (experiments) in the data. Shopify has both in spades and their Winter Edition 2026 release takes full advantage of this on behalf of their merchants. Big win for the entrepreneurs! Harley Finkelstein (full vid below)

The greatest constraint on exercising RL at scale on real world problems is the lack of data on end goals. Also, the lack of variation (experiments) in the data. Shopify has both in spades and their Winter Edition 2026 release takes full advantage of this on behalf of their merchants. Big win for the entrepreneurs! Harley Finkelstein (full vid below)

Ajay Agrawal

26,176 görüntüleme • 9 ay önce

im really done with this game! i play always 4-6hrs most of the time solo squad and dying a lot because of this shit, and no one cares about the problems in the game. PUBG: BATTLEGROUNDS you guys have to work on these stuff, this game is full stress and with these problems it getting worse!

im really done with this game! i play always 4-6hrs most of the time solo squad and dying a lot because of this shit, and no one cares about the problems in the game. PUBG: BATTLEGROUNDS you guys have to work on these stuff, this game is full stress and with these problems it getting worse!

BadBoyy2k

21,658 görüntüleme • 1 yıl önce

It's here. The first fully native agent harness, built for Windows. No electron in sight. Osaurus for Windows is almost ready for beta. Get ready for a team of agents running right on your PC, local and private. Watch Qwen3.8-27B set up a team of agents to perform a code review.

It's here. The first fully native agent harness, built for Windows. No electron in sight. Osaurus for Windows is almost ready for beta. Get ready for a team of agents running right on your PC, local and private. Watch Qwen3.8-27B set up a team of agents to perform a code review.

Osaurus

18,683 görüntüleme • 19 gün önce

“Cubana Chief Priest Has a Point About What He Said About Imo State. But What He Is Not Telling Us Is That Imo Is The Headquarters Of Prost!tut!on And Inf£ction In Nigeria. It’s Also The Headquarters Of Y@hoo-Y@hoo Culture, Has One Of The Highest Unemployment Rates, And Is a Hotspot For K!dnapping And Insecur!ty In The Entire Southeast. Imo Has One Of The Largest Gas Plants In West Africa, Yet There’s No Gas And Electricity In The State. But You’re Claiming You Generate Your Own Power.” ~ UK-Based Nigerian Godwin_Je Reacts

“Cubana Chief Priest Has a Point About What He Said About Imo State. But What He Is Not Telling Us Is That Imo Is The Headquarters Of Prost!tut!on And Inf£ction In Nigeria. It’s Also The Headquarters Of Y@hoo-Y@hoo Culture, Has One Of The Highest Unemployment Rates, And Is a Hotspot For K!dnapping And Insecur!ty In The Entire Southeast. Imo Has One Of The Largest Gas Plants In West Africa, Yet There’s No Gas And Electricity In The State. But You’re Claiming You Generate Your Own Power.” ~ UK-Based Nigerian Godwin_Je Reacts

Somto Okonkwo

34,665 görüntüleme • 1 ay önce