Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

So we know Gemma 4 is good at tool calling, but what about web coding? I threw 4 UI screenshots at three Gemma 4 models and said rebuild this, one shot, no hand-holding, just image in, code out. Model lineup: - E4B - 26B A4B (MoE) - 31B Dense... show more

stevibe

27,083 subscribers

124,839 просмотров • 3 месяцев назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

I gave two MoE models the same vibe coding challenge Qwen3.6 35B A3B (31.8GB) vs Gemma4 26B A4B (23.3GB) Stack: > Unsloth Q6_K_XL > llama.cpp > Model-card recommended sampling for each 4 prompts, side-by-side. Which one do you think wins?

I gave two MoE models the same vibe coding challenge Qwen3.6 35B A3B (31.8GB) vs Gemma4 26B A4B (23.3GB) Stack: > Unsloth Q6_K_XL > llama.cpp > Model-card recommended sampling for each 4 prompts, side-by-side. Which one do you think wins?

stevibe

261,895 просмотров • 2 месяцев назад

Parameter scaling just broke in front of me. I gave 90 math problems as images to 10 local vision models, 3 runs each, only consistent answers across all 3 runs counted. Two findings: > Gemma 4 was the most consistent family, 31B took the crown at 89.6%. > But Qwen 3.5 4B came 2 answers behind it. A 4B. Ranked 2nd out of 10. Vision-math isn't one skill, it's two: reading the image, then solving it. The real lesson for anyone running local: small ≠ weak. If you're building agentic workflows, finding the right model for each task matters more than reaching for the biggest one. In this test, the 4B ran far faster because of its size, scored higher, and left VRAM free for the rest of your stack. Full results: 🥇 Gemma 4 31B — 242/270 (89.6%) 🥈 Qwen 3.5 4B — 240/270 (88.9%) 🥉 Gemma 4 E4B — 222/270 (82.2%) 🥉 Qwen 3.6 27B — 222/270 (82.2%) 5. Gemma 4 26B A4B — 216/270 (80.0%) 6. Qwen 3.5 2B — 201/270 (74.4%) 7. Gemma 4 E2B — 192/270 (71.1%) 8. Qwen 3.6 35B A3B — 192/270 (71.1%) 9. Qwen 3.5 9B — 168/270 (62.2%) 10. Qwen 3.5 0.8B — 45/270 (16.7%) All GGUF + mmproj, Unsloth's Q6_K_XL quant.

Parameter scaling just broke in front of me. I gave 90 math problems as images to 10 local vision models, 3 runs each, only consistent answers across all 3 runs counted. Two findings: > Gemma 4 was the most consistent family, 31B took the crown at 89.6%. > But Qwen 3.5 4B came 2 answers behind it. A 4B. Ranked 2nd out of 10. Vision-math isn't one skill, it's two: reading the image, then solving it. The real lesson for anyone running local: small ≠ weak. If you're building agentic workflows, finding the right model for each task matters more than reaching for the biggest one. In this test, the 4B ran far faster because of its size, scored higher, and left VRAM free for the rest of your stack. Full results: 🥇 Gemma 4 31B — 242/270 (89.6%) 🥈 Qwen 3.5 4B — 240/270 (88.9%) 🥉 Gemma 4 E4B — 222/270 (82.2%) 🥉 Qwen 3.6 27B — 222/270 (82.2%) 5. Gemma 4 26B A4B — 216/270 (80.0%) 6. Qwen 3.5 2B — 201/270 (74.4%) 7. Gemma 4 E2B — 192/270 (71.1%) 8. Qwen 3.6 35B A3B — 192/270 (71.1%) 9. Qwen 3.5 9B — 168/270 (62.2%) 10. Qwen 3.5 0.8B — 45/270 (16.7%) All GGUF + mmproj, Unsloth's Q6_K_XL quant.

stevibe

22,882 просмотров • 2 месяцев назад

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

151,786 просмотров • 1 месяц назад

"122B has to be smarter than 27B" I showed 4 UI components to three Qwen3.5 models and asked them to recreate them from a screenshot alone: - 27B (dense) - 35B-A3B (MoE) - 122B-A10B (MoE) Same screenshot. Same prompt. Same task. Which one do you think nailed it?

"122B has to be smarter than 27B" I showed 4 UI components to three Qwen3.5 models and asked them to recreate them from a screenshot alone: - 27B (dense) - 35B-A3B (MoE) - 122B-A10B (MoE) Same screenshot. Same prompt. Same task. Which one do you think nailed it?

stevibe

101,043 просмотров • 3 месяцев назад

no prompt engineering, no agentic harness, no tool calls. just me being lazy in llama.cpp's web ui and gemma 4 31b dense taking the task seriously. i typed "create gpu marketplace cards with hardware specs and prices per hour" and the model went and coded this ui, one shot, navy bg, glassmorphism cards, neon accent buttons, realistic pricing tiers per architecture. it even wrote a "why this looks premium" explanation under the code. for context this is a q4 quant of google's 31b dense thinking model, running on a rtx 5090 mobile 24gb in the rog scar 18 at around 15 tok/s sustained, same vram tier as a 3090 or 4090 desktop, so whatever you see here translates directly to your card at home. the whole interaction was me not trying and the model reasoning harder than the prompt deserved. that tells me more about where local ai is at in april 2026 than any leaderboard score. next test drops gemma 4 into hermes agent, autonomous tool calling, multi step reasoning, real agentic loop instead of a chat window. let's see what the same model does when it gets the right environment. more experiments coming anon. octopus invaders queued. same hardware, different tasks, all published here on x and all translatable to your 24gb card. for now the video below shows it coding live, gpu going brrr.

no prompt engineering, no agentic harness, no tool calls. just me being lazy in llama.cpp's web ui and gemma 4 31b dense taking the task seriously. i typed "create gpu marketplace cards with hardware specs and prices per hour" and the model went and coded this ui, one shot, navy bg, glassmorphism cards, neon accent buttons, realistic pricing tiers per architecture. it even wrote a "why this looks premium" explanation under the code. for context this is a q4 quant of google's 31b dense thinking model, running on a rtx 5090 mobile 24gb in the rog scar 18 at around 15 tok/s sustained, same vram tier as a 3090 or 4090 desktop, so whatever you see here translates directly to your card at home. the whole interaction was me not trying and the model reasoning harder than the prompt deserved. that tells me more about where local ai is at in april 2026 than any leaderboard score. next test drops gemma 4 into hermes agent, autonomous tool calling, multi step reasoning, real agentic loop instead of a chat window. let's see what the same model does when it gets the right environment. more experiments coming anon. octopus invaders queued. same hardware, different tasks, all published here on x and all translatable to your 24gb card. for now the video below shows it coding live, gpu going brrr.

Sudo su

28,939 просмотров • 3 месяцев назад

Today, we’re launching Gemma 4, our most intelligent open models to date. Built with the same breakthrough technology as Gemini 3, Gemma 4 brings advanced reasoning to your personal hardware and devices. Here’s what Gemma 4 unlocks for developers: — Intelligence-per-parameter: Our 31B (Dense) and 26B (MoE) models deliver state-of-the-art performance for their size, outcompeting models 20x their size on Arena.ai — Commercial flexibility: Released under a permissive Apache 2.0 license for complete developer flexibility and digital sovereignty — Agentic workflows: Native support for function-calling and structured JSON output allows you to build reliable, autonomous agents — Multimodal edge AI: The E2B and E4B models bring native vision, audio, and low latency to mobile and IoT devices — Long-context reasoning: Up to 256K context windows allow you to process entire repositories or large documents in a single prompt Whether you're building global applications in 140+ languages or local-first AI code assistants, Gemma 4 is built to be your foundation. Explore in Google AI Studio or download the weights on Hugging Face, Kaggle, and ollama.

Today, we’re launching Gemma 4, our most intelligent open models to date. Built with the same breakthrough technology as Gemini 3, Gemma 4 brings advanced reasoning to your personal hardware and devices. Here’s what Gemma 4 unlocks for developers: — Intelligence-per-parameter: Our 31B (Dense) and 26B (MoE) models deliver state-of-the-art performance for their size, outcompeting models 20x their size on Arena.ai — Commercial flexibility: Released under a permissive Apache 2.0 license for complete developer flexibility and digital sovereignty — Agentic workflows: Native support for function-calling and structured JSON output allows you to build reliable, autonomous agents — Multimodal edge AI: The E2B and E4B models bring native vision, audio, and low latency to mobile and IoT devices — Long-context reasoning: Up to 256K context windows allow you to process entire repositories or large documents in a single prompt Whether you're building global applications in 140+ languages or local-first AI code assistants, Gemma 4 is built to be your foundation. Explore in Google AI Studio or download the weights on Hugging Face, Kaggle, and ollama.

Google AI

1,669,327 просмотров • 3 месяцев назад

Round 2: Dense edition Qwen3.5 27B vs Gemma4 31B Same 4 HTML canvas challenges, this time no MoE. Setup: > Quant: Unsloth Q6_K_XL (both) > Each running on an RTX 5090 > Single HTML file, no libraries, real-time animation My read per prompt: 🌳 Tree — Gemma, branches are beautifully structured ✨ Night sky — close, but Qwen's stars blink better 🐟 Fish boids — Qwen, detailed fish and look at that motion, so realistic 🏠 House build — neither really nailed it Agree? Does dense actually beat MoE here?

Round 2: Dense edition Qwen3.5 27B vs Gemma4 31B Same 4 HTML canvas challenges, this time no MoE. Setup: > Quant: Unsloth Q6_K_XL (both) > Each running on an RTX 5090 > Single HTML file, no libraries, real-time animation My read per prompt: 🌳 Tree — Gemma, branches are beautifully structured ✨ Night sky — close, but Qwen's stars blink better 🐟 Fish boids — Qwen, detailed fish and look at that motion, so realistic 🏠 House build — neither really nailed it Agree? Does dense actually beat MoE here?

stevibe

33,304 просмотров • 2 месяцев назад

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

raullen

22,680 просмотров • 1 месяц назад

You can run Google new Gemma 4 on mobile easily. I am using Gemma 4 version E2B on my Pixel 10 Pro. Here is all you need to do: - Go to the App Store and install Google AI Edge Gallery. If you already have it, just update it. - From there, you can install the model directly and start using it. Still exploring what this small but powerful model can do :)

You can run Google new Gemma 4 on mobile easily. I am using Gemma 4 version E2B on my Pixel 10 Pro. Here is all you need to do: - Go to the App Store and install Google AI Edge Gallery. If you already have it, just update it. - From there, you can install the model directly and start using it. Still exploring what this small but powerful model can do :)

AshutoshShrivastava

226,336 просмотров • 3 месяцев назад

Yesterday we found out Qwen3.5-27B (dense) beats both the 35B and 122B MoE variants at UI replication from screenshots. Today's question: what about Jackrong's Qwen3.5-27B-Claude-Opus-4.6-Reasoning-Distilled? A distilled 27B with Claude Opus 4.6 reasoning baked in. So I put all 3 in the ring: - Qwen3.5-27B - Qwen3.5-27B-Claude-Opus-Distilled - Claude Opus 4.6 Since the distilled model isn't multimodal, I gave all three the same detailed text prompts describing the UI components from yesterday's test.

Yesterday we found out Qwen3.5-27B (dense) beats both the 35B and 122B MoE variants at UI replication from screenshots. Today's question: what about Jackrong's Qwen3.5-27B-Claude-Opus-4.6-Reasoning-Distilled? A distilled 27B with Claude Opus 4.6 reasoning baked in. So I put all 3 in the ring: - Qwen3.5-27B - Qwen3.5-27B-Claude-Opus-Distilled - Claude Opus 4.6 Since the distilled model isn't multimodal, I gave all three the same detailed text prompts describing the UI components from yesterday's test.

stevibe

125,913 просмотров • 3 месяцев назад

I wanted to make a portfolio site but I didn't want to make a lame one so I spent the last ~4 months learning web dev and Three.js from scratch to make this really pretty one try it out and let me know what you think (link in replies) #threejs #webgpu #shaders

I wanted to make a portfolio site but I didn't want to make a lame one so I spent the last ~4 months learning web dev and Three.js from scratch to make this really pretty one try it out and let me know what you think (link in replies) #threejs #webgpu #shaders

Swifter

12,971 просмотров • 4 месяцев назад

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99 flags. same 48k context. unsloth gemma-4-12b-it-Q4_K_M.gguf → 15 tok/sec @ 48k ctx unsloth gemma-4-12B-it-qat-UD-Q4_K_XL.gguf → 32 tok/sec @ 48k ctx → 26 tok/sec @ 64k ctx 64k context is a big deal. Hermes 3 agent requires 64k minimum to run. you're now getting full hermes compatible context on a budget consumer GPU at 26 tok/sec locally. 2.1x faster on identical hardware. and here's the part that breaks your brain: the QAT-UD-Q4_K_XL is actually SMALLER than the Q4_K_M "XL" why? QAT = Quantization Aware Training Google didn't train the model first and compress it later they trained it to be quantized from day one the weights already know how to survive low precision that's why you get more quality per byte llamacpp flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -cnv -ngl 99 -c 48000 -v fits in 8GB VRAM clean. no API. no cloud. no subscription. and this isn't even the MTP variant yet Gemma-4-E2B QAT runs on 3GB RAM, E4B on 5GB, 12B on 7GB, 26-A4B on 15GB and 31B on 18GB. I have benchmarked the 26b and 31b qat as well on a single RTX 4090, checkout the comments for details. If you have a 6GB or 8GB VRAM GPU, post your numbers. more benchmarks and configs coming soon

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99 flags. same 48k context. unsloth gemma-4-12b-it-Q4_K_M.gguf → 15 tok/sec @ 48k ctx unsloth gemma-4-12B-it-qat-UD-Q4_K_XL.gguf → 32 tok/sec @ 48k ctx → 26 tok/sec @ 64k ctx 64k context is a big deal. Hermes 3 agent requires 64k minimum to run. you're now getting full hermes compatible context on a budget consumer GPU at 26 tok/sec locally. 2.1x faster on identical hardware. and here's the part that breaks your brain: the QAT-UD-Q4_K_XL is actually SMALLER than the Q4_K_M "XL" why? QAT = Quantization Aware Training Google didn't train the model first and compress it later they trained it to be quantized from day one the weights already know how to survive low precision that's why you get more quality per byte llamacpp flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -cnv -ngl 99 -c 48000 -v fits in 8GB VRAM clean. no API. no cloud. no subscription. and this isn't even the MTP variant yet Gemma-4-E2B QAT runs on 3GB RAM, E4B on 5GB, 12B on 7GB, 26-A4B on 15GB and 31B on 18GB. I have benchmarked the 26b and 31b qat as well on a single RTX 4090, checkout the comments for details. If you have a 6GB or 8GB VRAM GPU, post your numbers. more benchmarks and configs coming soon

Alok

259,993 просмотров • 1 месяц назад

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Prince Canuma

82,169 просмотров • 3 месяцев назад

T 5168 - YO ! I created these 4 designs myself .. YES I did !! So here’s what I want from you .. ! Which do you think is the best one ? कौन सा ?? बताइए Let me know .. #DesignWithAB #GetMacmerised

T 5168 - YO ! I created these 4 designs myself .. YES I did !! So here’s what I want from you .. ! Which do you think is the best one ? कौन सा ?? बताइए Let me know .. #DesignWithAB #GetMacmerised

Amitabh Bachchan

98,041 просмотров • 1 год назад

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasn’t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10–12B parameters, 3GB RAM). Video credit: Google YT

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasn’t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10–12B parameters, 3GB RAM). Video credit: Google YT

AshutoshShrivastava

210,490 просмотров • 1 год назад

🚨 One orchestrator. 10 parallel agents. 100+ tokens a second. All local. The Google Gemma team just dropped a MASSIVE demo for Gemma 4 26B. They built a concurrent workflow that lets the 26B model coordinate an entire team of sub-agents on your machine. Out of the box, the cookbook lets you run 10 parallel agents to: → Code an entire SVG art gallery in seconds → Translate text simultaneously → Generate ASCII art → Write parallel code Spinning up multi-agent systems locally has never looked this fast or this accessible. 100% free and open-source. repo link in 🧵↓

🚨 One orchestrator. 10 parallel agents. 100+ tokens a second. All local. The Google Gemma team just dropped a MASSIVE demo for Gemma 4 26B. They built a concurrent workflow that lets the 26B model coordinate an entire team of sub-agents on your machine. Out of the box, the cookbook lets you run 10 parallel agents to: → Code an entire SVG art gallery in seconds → Translate text simultaneously → Generate ASCII art → Write parallel code Spinning up multi-agent systems locally has never looked this fast or this accessible. 100% free and open-source. repo link in 🧵↓

Charly Wargnier

31,627 просмотров • 29 дней назад

No bad chemistry here! The DEA C.L.E.T drops this Friday at 4:30pm ET! Watch the video for full specs and let us know what you think in the comments.

No bad chemistry here! The DEA C.L.E.T drops this Friday at 4:30pm ET! Watch the video for full specs and let us know what you think in the comments.

Palmetto State Armory

28,121 просмотров • 8 месяцев назад

🔥 Comparison of 4 AI Image Generators 🎨 I put 4 AI models to the test with 5 different prompts, running 4 attempts each Here are the contenders: 🔹 Google Imagen 3 🔹 Mystic 2.5 🔹 Photon 🔹Frames Each model interpreted the prompts in its own way, producing unique results. Which one do you think did it best? Prompts below Let me know in the comments! 👇

🔥 Comparison of 4 AI Image Generators 🎨 I put 4 AI models to the test with 5 different prompts, running 4 attempts each Here are the contenders: 🔹 Google Imagen 3 🔹 Mystic 2.5 🔹 Photon 🔹Frames Each model interpreted the prompts in its own way, producing unique results. Which one do you think did it best? Prompts below Let me know in the comments! 👇

MayorkingAI

31,939 просмотров • 1 год назад

$A $351 MINI PC IS RUNNING 26-BILLION-PARAMETER AI MODELS AT 20 TOKENS/SEC AND HERMES AGENT ON TOP OF IT This is the Minisforum UM790 Pro. AMD Ryzen 9 7940HS, Radeon 780M iGPU, 48GB DDR5. The BIOS reports the GPU has 4GB of VRAM Here's the part people get wrong. The 780M has no dedicated VRAM at all it borrows from system RAM. Vulkan ignores the BIOS number and reads the full 48GB pool directly That's the whole trick. 21+ GB allocated to model weights on a "4GB" GPU. Because the models are MoE, only 4 billion of the parameters activate per token a fraction of the reads a dense model needs. That's why 20 tok/s works here Gemma 4 26B MoE holds 19.5 tok/s with 196K context. Qwen3.5-35B-A3B holds 20.8. Nemotron Cascade 2 clears 24.8. A dense 31B, by contrast, drops to 4 tok/s it reads the entire model every token, no way around it On top: Hermes Agent, full agentic workflows terminal, file ops, web, 40+ tools against local models only. No API keys. The wall between you and a usable local agent used to be a GPU you couldn't afford. Now it's a BIOS setting most people never check Bookmark this & Try it yourself ↓$

A $351 MINI PC IS RUNNING 26-BILLION-PARAMETER AI MODELS AT 20 TOKENS/SEC AND HERMES AGENT ON TOP OF IT This is the Minisforum UM790 Pro. AMD Ryzen 9 7940HS, Radeon 780M iGPU, 48GB DDR5. The BIOS reports the GPU has 4GB of VRAM Here's the part people get wrong. The 780M has no dedicated VRAM at all it borrows from system RAM. Vulkan ignores the BIOS number and reads the full 48GB pool directly That's the whole trick. 21+ GB allocated to model weights on a "4GB" GPU. Because the models are MoE, only 4 billion of the parameters activate per token a fraction of the reads a dense model needs. That's why 20 tok/s works here Gemma 4 26B MoE holds 19.5 tok/s with 196K context. Qwen3.5-35B-A3B holds 20.8. Nemotron Cascade 2 clears 24.8. A dense 31B, by contrast, drops to 4 tok/s it reads the entire model every token, no way around it On top: Hermes Agent, full agentic workflows terminal, file ops, web, 40+ tools against local models only. No API keys. The wall between you and a usable local agent used to be a GPU you couldn't afford. Now it's a BIOS setting most people never check Bookmark this & Try it yourself ↓

slash1s

35,742 просмотров • 28 дней назад