Loading video...

Video Failed to Load

Go Home

Moondream 3 Preview adds frontier-level visual reasoning while retaining Moondream 2's blazing speed. No compromises.

44,461 views • 1 year ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

HydraFusion Explained. Part I: How does the Copilot engine know what to optimize for? Your prompt is evaluated across 4 dimensions: ➡ Does it require deep reasoning? (aka. reasoning depth) ➡ Is it a sophisticated problem? (aka. code generation complexity) ➡ Is it untangling a complicated mess? (aka. debugging difficulty) ➡ Is it dominated by tool-use? (aka. tool orchestration needs) Based on this evaluation, a HyDRA score is assigned to determine the capability profile your task needs the most and to establish a quality bar. Part II: How does it choose a model? Note: It doesn' t pick one model to handle the entire job e2e, (that's Auto mode). Instead, it selects 1 of 3 execution workflows and assigns the best model at different stages based on the HyDRA score: 1️⃣ Single ⚙️ How it works: A single model completes the task from start to finish. ⚖️ Rationale: The task comfortably meets the quality bar with one model. Multi-model orchestration would add latency and cost with no meaningful quality gain. 2️⃣ Cascade ⚙️ How it works: A lightweight, cost-efficient model generates the solution. This draft is evaluated against a quality gate and if it falls short of the quality bar, the entire task escalates to a stronger, frontier model. ⚖️ Rationale: Only bring in the big guns when there is concrete evidence that a lightweight model won't meet the quality threshold. 3️⃣ Critique ⚙️ How it works: A lightweight model drafts the initial code and tool interactions. An independent, read-only frontier model reviews that draft and provides feedback. The original lightweight model then performs any targeted revision(s) before the final response is sent to the user. ⚖️ Rationale: Writing code (output tokens) is expensive while reviewing code (input tokens) is cheap. Instead of incurring the cost of a powerhouse writing hundreds of lines from scratch, a cost-efficient model writes the first draft, and the frontier model just reviews it and points out fixes. HydraFusion is available in experimental preview on the GitHub Copilot CLI: /experimental on, /model and select Hydrafusion (Research Preview)

Julia Muiruri

13,000 views • 1 month ago

50% more context unlocked for Qwen 3.8 27b Q4_K_XL dflash 2 on a single RTX 4090 (24 GB VRAM) I found a hidden VRAM tax in llama.cpp. By combining my custom 2 bit DFlash 2 drafter with one overlooked server flag, I just unlocked another +80,000 tokens of context. Qwen3.8-27B is now running a massive 250,000 context at 75 tokens/s on a single RTX 4090. Here is the secret: By default, `llama-server` reserves massive chunks of your VRAM to handle multiple concurrent users (batching). If you are running a single user session, you are bleeding memory for features you aren't using. By passing the `--parallel 1` flag, you force the engine to dedicate 100% of your 24GB VRAM buffer to a single user. When we combine the VRAM saved by our Q2_K 2-bit drafter with the VRAM saved by `--parallel 1`, the context ceilings absolutely explode: Note: all benchmarks carried out with a massive 28k prompt. Ubuntu 22. ### THE NEW 24GB PHYSICAL LIMITS (Single RTX 4090): # 1. The "Repo Swallower" (Q4 KV Cache): - Context: 250,000 tokens (Up from 170k!) - Speed: 73.66 t/s decode | 1,608 t/s prefill - Peak VRAM: 23.8 GB # 2. The "High-Precision SWE" (Q8 KV Cache): - Context: 150,000 tokens (Up from 100k!) - Speed: 75.01 t/s decode | 1,667 t/s prefill - Peak VRAM: 23.9 GB # 3. The "Pristine Attention" (Unquantized FP16 KV): - Context: 90,000 tokens - Speed: 80.58 t/s decode | 1,699 t/s prefill - Peak VRAM: 23.92 GB ### HOW TO RUN THE 250K GOD STACK TODAY: (Requires PR #27342 + my Q2_K Hugging Face drafter) llama.cpp flags: ./build/bin/llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -md Qwen3.8-27B-DFlash2-Q2_K.gguf --spec-type draft-dflash --spec-draft-n-max 3 -c 250000 -ngl 99 --parallel 1 --port 8080 -ctv q4_0 -ctk q4_0 We are pushing a quarter million tokens of context with speculative DFlash 2 decoding at 73 tokens/second on a single consumer gaming GPU. I dropped my custom 2 bit Hugging Face GGUF links, visual performance graphs, and the PR #27342 build instructions in the replies below. If you own a single RTX 3090 or 4090, it is officially time to cancel your API subscriptions and let local silicon eat the cloud. how much monthly API spend does an optimized 4090 rig like this actually replace for you?

Alok

41,735 views • 2 months ago

Qwen3.8-Flash-Next is starting to feel like the local model Opus fans have been waiting for. Someone ran the NVFP4 176B-class Flash-Next on 2× DGX Sparks, and the results are wild. Real measured scaling → C1: 44.2 tok/s → C2: 64.6 tok/s → C4: 86.8 tok/s aggregate The per-stream speed drops with concurrency, but total throughput keeps climbing. Long-context behavior was even more impressive: → 5K: needle retrieved → 21K: needle retrieved → 84K: needle retrieved → 167K: needle retrieved → 262K: prefill succeeded, but the window was saturated That 167K retrieval test is the one I care about. Long agent runs are where models usually start losing the plot. Flash-Next didn’t. It also held up surprisingly well on physics-heavy reasoning, artifact generation, research workflows, evidence checking, and long-horizon planning. The personality is interesting too. DeepSeek V4 Flash feels like the dependable workhorse. GLM-5.2 feels like the problem-solving machine. Qwen3.8-Flash-Next feels more insightful. It has that rare ability to understand what you’re actually asking rather than just following the surface pattern. The main weakness I’ve noticed is instruction following. It can occasionally drift between prose turns where DeepSeek and GLM stay tighter. And this is why the 256GB M5 Ultra conversation gets interesting. If Apple can pair that huge unified-memory pool with enough bandwidth, this model class becomes genuinely practical for long-running local agents. We’re talking about frontier-class reasoning on hardware sitting on a desk.

FHILY👑

20,253 views • 1 month ago