Loading video...

Video Failed to Load

Go Home

Baidu just open-sourced an OCR model that reads entire 40-page documents in one shot. It's called Unlimited-OCR. 3 billion parameters but only 500 million active during inference. Runs 100% locally on your machine. Why this matters: traditional OCR tools chop documents page by page. Tables that span two pages...

VaibhavSisinty's profile picture

Vaibhav Sisinty

170,828 subscribers

415,499 views • 1 month ago •via X (Twitter)

Science & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.
0:15

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.

AlphaSignal

92,137 views • 5 months ago

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.
0:15

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.

Jafar Najafov

13,630 views • 5 months ago

1.7 billion free tokens per month. A month ago i showed you how to route claude code through free providers. someone just shipped the cleanest version of this setup yet… it's called Freellmapi 13,400+ stars on github, MIT licensed, takes 2 minutes to install. what it does: stacks the free tiers of 16 different LLM providers behind one local API. point claude code, codex, or cursor at that one endpoint, and it automatically routes your calls across all 16 free pools. The 16 providers it covers: Google, Groq, Cerebras, Mistral, OpenRouter, GitHub Models, Cloudflare, Cohere, NVIDIA, HuggingFace, Ollama Cloud, Kilo, Pollinations, LLM7, OVH, and OpenCode Zen. if you sign up to all 16 and add your free API keys, you get roughly 1.7 billion free tokens per month combined. ▫️ How to install (one command) curl -fsSL bash this runs the whole thing locally on your machine through Docker. once it's up, open paste your provider keys on the Keys page, and grab the unified API key from the dashboard. that's the key you point your apps at. With this, claude code stops hitting your monthly cap because every prompt routes through the 16 free pools instead of your paid plan. and if one provider rate-limits mid-conversation, freellmapi falls over to the next one automatically so your session never breaks. repo: Free, MIT-licensed, runs on your laptop or a $5 VPS.
0:18

1.7 billion free tokens per month. A month ago i showed you how to route claude code through free providers. someone just shipped the cleanest version of this setup yet… it's called Freellmapi 13,400+ stars on github, MIT licensed, takes 2 minutes to install. what it does: stacks the free tiers of 16 different LLM providers behind one local API. point claude code, codex, or cursor at that one endpoint, and it automatically routes your calls across all 16 free pools. The 16 providers it covers: Google, Groq, Cerebras, Mistral, OpenRouter, GitHub Models, Cloudflare, Cohere, NVIDIA, HuggingFace, Ollama Cloud, Kilo, Pollinations, LLM7, OVH, and OpenCode Zen. if you sign up to all 16 and add your free API keys, you get roughly 1.7 billion free tokens per month combined. ▫️ How to install (one command) curl -fsSL bash this runs the whole thing locally on your machine through Docker. once it's up, open paste your provider keys on the Keys page, and grab the unified API key from the dashboard. that's the key you point your apps at. With this, claude code stops hitting your monthly cap because every prompt routes through the 16 free pools instead of your paid plan. and if one provider rate-limits mid-conversation, freellmapi falls over to the next one automatically so your session never breaks. repo: Free, MIT-licensed, runs on your laptop or a $5 VPS.

Axel Bitblaze 🪓

48,825 views • 2 months ago

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.
0:16

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.

elvis

11,303 views • 23 days ago

you're paying $20/mo for something your $500 GPU can already do. Gemma 4 26B A4B QAT MoE + Hermes Agent running on a single RTX 4060 (8GB VRAM). Built a vision capable, 100% free, 100% local, private AI assistant that lives in my Chrome browser. No API keys. No cloud. No subscriptions. 100% vibe coded. 0% handholding. It has full context of whatever's on my screen can answer questions, summarize pages, extract data, and see images. Same local model handles everything, no external calls, ever. keep reading for the model and hermes agent tips i learnt while building this locally. Here's the exact setup for anyone running local LLMs on 6-8 GB VRAM: llama.cpp server flags (on my NVIDIA RTX 4060 8gb VRAM): -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --cache-type-k q8_0 --cache-type-v q8_0 -c 150000 --port 8080 Throughput with quantization: Prefill: 200-250 tokens/sec Decode: 20-25 tokens/sec reduce context if oom on 6 gb vram card. Key learnings: - Quantize KV cache to q8 for faster prefill/decode. Prefill goes from 100-150 (unquantized) to 200-250 tok/s (q8). - But watch out, once actual context grows past ~50k tokens on high entropy workloads, q8 KV quantization can cause hallucinations. Low entropy workloads are mostly unaffected. If you see it happening, drop the quantization. This is common across all local models. - In Hermes Agent settings -> Memory & Context, bump compression threshold from default 0.5 to 0.7. Default triggers way too frequent context compression and eats time. Up next: add persistent memory, web search, tool calling, streaming output and whatever you suggest. Running a 26B MoE with vision + 150k context window on 8GB VRAM would've sounded impossible 6 months ago. Works the same on the NVIDIA RTX 3060 Ti, 3070, 4060 Ti, 5060, 2080, or any 8GB card. VRAM is the only requirement. Local AI agents are closer than people think. You just need to know where the knobs are. Model's Unsloth quant hugging face link in the comments. Have you tried Hermes agent by Nous Research yet? What are you building with local LLMs? Drop it below, let's see what this community is shipping.
0:19

you're paying $20/mo for something your $500 GPU can already do. Gemma 4 26B A4B QAT MoE + Hermes Agent running on a single RTX 4060 (8GB VRAM). Built a vision capable, 100% free, 100% local, private AI assistant that lives in my Chrome browser. No API keys. No cloud. No subscriptions. 100% vibe coded. 0% handholding. It has full context of whatever's on my screen can answer questions, summarize pages, extract data, and see images. Same local model handles everything, no external calls, ever. keep reading for the model and hermes agent tips i learnt while building this locally. Here's the exact setup for anyone running local LLMs on 6-8 GB VRAM: llama.cpp server flags (on my NVIDIA RTX 4060 8gb VRAM): -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --cache-type-k q8_0 --cache-type-v q8_0 -c 150000 --port 8080 Throughput with quantization: Prefill: 200-250 tokens/sec Decode: 20-25 tokens/sec reduce context if oom on 6 gb vram card. Key learnings: - Quantize KV cache to q8 for faster prefill/decode. Prefill goes from 100-150 (unquantized) to 200-250 tok/s (q8). - But watch out, once actual context grows past ~50k tokens on high entropy workloads, q8 KV quantization can cause hallucinations. Low entropy workloads are mostly unaffected. If you see it happening, drop the quantization. This is common across all local models. - In Hermes Agent settings -> Memory & Context, bump compression threshold from default 0.5 to 0.7. Default triggers way too frequent context compression and eats time. Up next: add persistent memory, web search, tool calling, streaming output and whatever you suggest. Running a 26B MoE with vision + 150k context window on 8GB VRAM would've sounded impossible 6 months ago. Works the same on the NVIDIA RTX 3060 Ti, 3070, 4060 Ti, 5060, 2080, or any 8GB card. VRAM is the only requirement. Local AI agents are closer than people think. You just need to know where the knobs are. Model's Unsloth quant hugging face link in the comments. Have you tried Hermes agent by Nous Research yet? What are you building with local LLMs? Drop it below, let's see what this community is shipping.

Alok

36,691 views • 2 months ago

A 19-YEAR-OLD FROM CANADA TURNED TWERK CLIPS INTO A $6,800/MONTH AI PAGE. Mint hair. Back turned to the camera. Soft daylight, a balcony view, and 12 seconds of movement built almost entirely around one thing: scroll-stopping motion. She dances, twerks, runs her hands through her hair, and gives the kind of clip that people replay before they even decide whether she is real. That is the whole edge. The operator behind the page understood that in this niche, the first win is not storytelling, it is interruption. If the movement stops the thumb for half a second, the algorithm does the rest. The page is run like a funnel, not like a normal content account. Short clips like this go out free on Instagram and TikTok, usually 3–5 times a day, with the best performers pushed again in slightly different versions. One character, one recognizable look, one consistent vibe. The traffic then gets pushed into a paid page where subscriptions sit between $9.99 and $16.99, with extra revenue coming from tips and locked content. On numbers like these, a page can realistically stack up to around 410 paying subs at an average of $11.50, plus $1,900 in PPV and tips, which is how a single AI girl built around dance content gets to roughly $6,800/month. What makes this work is not just that she is attractive. Thousands of AI girls are attractive now. What works is that the content is built for rewatch. Twerking clips, especially from a back-facing angle like this, are simple, fast, and instantly understood by the viewer. There is no setup needed. No explanation. No context. Just motion, attention, and curiosity. People replay it, send it, comment on whether she is real, and that creates the exact kind of engagement these pages live on. The catch, as always, is retention. A clip like this can bring in the spike, but the money only stays if the character keeps posting and keeps feeling consistent. That is why the smarter operators lock the identity, keep the same hair, same body language, same style, and turn one good-performing angle into a repeatable system. One AI girl, one signature dance style, and one content loop that turns short clips into about $6.8K a month.
0:11

A 19-YEAR-OLD FROM CANADA TURNED TWERK CLIPS INTO A $6,800/MONTH AI PAGE. Mint hair. Back turned to the camera. Soft daylight, a balcony view, and 12 seconds of movement built almost entirely around one thing: scroll-stopping motion. She dances, twerks, runs her hands through her hair, and gives the kind of clip that people replay before they even decide whether she is real. That is the whole edge. The operator behind the page understood that in this niche, the first win is not storytelling, it is interruption. If the movement stops the thumb for half a second, the algorithm does the rest. The page is run like a funnel, not like a normal content account. Short clips like this go out free on Instagram and TikTok, usually 3–5 times a day, with the best performers pushed again in slightly different versions. One character, one recognizable look, one consistent vibe. The traffic then gets pushed into a paid page where subscriptions sit between $9.99 and $16.99, with extra revenue coming from tips and locked content. On numbers like these, a page can realistically stack up to around 410 paying subs at an average of $11.50, plus $1,900 in PPV and tips, which is how a single AI girl built around dance content gets to roughly $6,800/month. What makes this work is not just that she is attractive. Thousands of AI girls are attractive now. What works is that the content is built for rewatch. Twerking clips, especially from a back-facing angle like this, are simple, fast, and instantly understood by the viewer. There is no setup needed. No explanation. No context. Just motion, attention, and curiosity. People replay it, send it, comment on whether she is real, and that creates the exact kind of engagement these pages live on. The catch, as always, is retention. A clip like this can bring in the spike, but the money only stays if the character keeps posting and keeps feeling consistent. That is why the smarter operators lock the identity, keep the same hair, same body language, same style, and turn one good-performing angle into a repeatable system. One AI girl, one signature dance style, and one content loop that turns short clips into about $6.8K a month.

Rimi

13,962 views • 17 days ago

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.
0:10

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,842,664 views • 4 months ago

I just ran Gemma 4 31B on @CerebrasSystems at 1,800+ tokens/sec and it's multimodal. For context: that's 35x faster than a typical GPU endpoint, and the first token (reasoning included) lands in 1.5 seconds. This isn't a benchmark slide, I recorded the inference live. Prompt I used: "Create a simulation of an iPhone. Include at least one working dummy note taking app, a functional notification pulldown, high quality graphics, single HTML file, any libs via CDN." - Generation time: 3 seconds. - Notes app worked. - Notification panel worked. - Rendered first try. This is what wafer-scale inference unlocks, not just "faster," but a different category of product. When generation is this fast, you stop waiting and start iterating in real time. Why this matters: Gemma 4 31B is Google DeepMind's flagship open weight model, Apache 2.0 licensed, dense (not MoE), and built for efficiency over raw parameter count. It scores close to Claude Haiku 4.5 on the Artificial Analysis Intelligence Index (30 vs 29) but runs ~18x faster on Cerebras. It's also the first multimodal model on Cerebras's platform, meaning you can now feed it screenshots, documents, charts, and UI states at wafer scale speed. # Applications I'm most excited about: - Screenshot → Insight: Drop in a dashboard or document screenshot, get structured findings back instantly. no waiting, no batching. - Live UI generation: Full interactive interfaces (like my iPhone sim) generated and rendered in under 2 seconds. - Screenshot -> Patch: Feed it a broken UI + console error, get a minimal code fix and verification steps back. - Computer use & agentic loops: See -> reason -> act - verify, fast enough to keep a human in the loop instead of waiting on the model. - Long context summarization: Full research reports condensed into decision ready summaries you can read and requery in one sitting. The bigger unlock isn't the speed number itself, it's that agentic and multimodal loops (see -> reason -> output -> tool call -> verify -> retry) finally run in real time instead of feeling sluggish. As Logan Kilpatrick (Logan Kilpatrick) put it: "If every model was doing 2,000 tokens per second, you wouldn't build the same product and just have it be faster, you'd build different products." Gemma 4 31B is live now on Cerebras Inference Cloud in public preview. If you're building multimodal, agentic, or real time apps, this is worth testing today. What would you build with such insane inference throughput?
0:12

I just ran Gemma 4 31B on @CerebrasSystems at 1,800+ tokens/sec and it's multimodal. For context: that's 35x faster than a typical GPU endpoint, and the first token (reasoning included) lands in 1.5 seconds. This isn't a benchmark slide, I recorded the inference live. Prompt I used: "Create a simulation of an iPhone. Include at least one working dummy note taking app, a functional notification pulldown, high quality graphics, single HTML file, any libs via CDN." - Generation time: 3 seconds. - Notes app worked. - Notification panel worked. - Rendered first try. This is what wafer-scale inference unlocks, not just "faster," but a different category of product. When generation is this fast, you stop waiting and start iterating in real time. Why this matters: Gemma 4 31B is Google DeepMind's flagship open weight model, Apache 2.0 licensed, dense (not MoE), and built for efficiency over raw parameter count. It scores close to Claude Haiku 4.5 on the Artificial Analysis Intelligence Index (30 vs 29) but runs ~18x faster on Cerebras. It's also the first multimodal model on Cerebras's platform, meaning you can now feed it screenshots, documents, charts, and UI states at wafer scale speed. # Applications I'm most excited about: - Screenshot → Insight: Drop in a dashboard or document screenshot, get structured findings back instantly. no waiting, no batching. - Live UI generation: Full interactive interfaces (like my iPhone sim) generated and rendered in under 2 seconds. - Screenshot -> Patch: Feed it a broken UI + console error, get a minimal code fix and verification steps back. - Computer use & agentic loops: See -> reason -> act - verify, fast enough to keep a human in the loop instead of waiting on the model. - Long context summarization: Full research reports condensed into decision ready summaries you can read and requery in one sitting. The bigger unlock isn't the speed number itself, it's that agentic and multimodal loops (see -> reason -> output -> tool call -> verify -> retry) finally run in real time instead of feeling sluggish. As Logan Kilpatrick (Logan Kilpatrick) put it: "If every model was doing 2,000 tokens per second, you wouldn't build the same product and just have it be faster, you'd build different products." Gemma 4 31B is live now on Cerebras Inference Cloud in public preview. If you're building multimodal, agentic, or real time apps, this is worth testing today. What would you build with such insane inference throughput?

Alok

12,962 views • 2 months ago

I am the Under Secretary for Strategic Resource Integration. We sit inside the Executive Office of the President. You won't find us on an org chart. We requested it that way. You think an office like mine spins up in a crisis. We've been open since January. Venezuela wasn't a war. Venezuela was the beta. Maduro proved the model the morning we took delivery of him. The President called it "working out brilliantly for both." He's right. It works brilliantly for both of us, and the customer will come around to the "both" on their own time. Kharg Island is the rollout. A third the size of Manhattan, and 90 percent of Iran's crude runs through it. 2 million barrels a day, one valve. I have prayed my whole career for a country that legible. 53 billion in revenue. 11 percent of their GDP. All of it funneling through a rock you could walk across before lunch. I kept the Venezuela playbook. I changed the classification banner and let find-and-replace do Caracas to Tehran. That was the plan. That was the whole plan. Page 14 still says Caracas. No one on the distribution list has ever reached page 14. They initial page 1. Page 1 is projected revenue. It goes up and to the right, the only direction this town can read. The Energy Secretary testified he was "unaware." Under oath, to Congress, the man who runs American energy. Good. He was never read into the compartment. The fewer names on the account, the cleaner the account. He runs the department. He does not run this. We don't say invasion. We say onboarding. Every onboarding has friction. The Marines off the coast are not an occupying force. They're field engagement. Iran said an invasion of its islands would "shatter all restraint." I logged it as customer feedback and filed it. The file is full. Engaged customers submit the most feedback. It's the quiet ones who churn. And Iran can't churn. That's the part I'm proudest of. We built the account with no exit. No competitor to switch to. We are the market, and the market is now total. Total is my word. I workshopped it. "Seize" scares people. "Total control of their markets" tests at 71, because nobody pictures a kid from Ohio on a beach when you say markets. We moved 22 tankers through Hormuz at night with the lights off. The President said he wanted to announce it so badly it hurt, and he held it a month anyway. That is the most discipline this administration has ever shown. 100 million barrels, he said. The fact-checkers say the math doesn't hold. They're junior. The number was never the deliverable. The number is the soft launch. He said he loves the inflation. People clutched their receipts. 90 a barrel instead of 250 is the discount, and 250 is a price I invented so this one would feel like mercy. How long will we be there? The President answered that. "We had to be there for a while." A while is my favorite contract length. It renews on its own. No court can find the end date. He wondered out loud whether America has the stomach for it. I don't need the country's stomach. I need its signature on page 1. He called the people who object stupid. I'd defend them. They're not stupid. They just read to page 14. Last thing. Annex C. The out-year targets. 3 more, redacted, marked pre-decisional. I can't tell you which 3. Total control of their markets. Nobody in that room asked which market. The crude still sails east, to China, the same as yesterday. We don't own the oil. We own the valve, the fear, and the word "total." That was always the only product.
0:18

I am the Under Secretary for Strategic Resource Integration. We sit inside the Executive Office of the President. You won't find us on an org chart. We requested it that way. You think an office like mine spins up in a crisis. We've been open since January. Venezuela wasn't a war. Venezuela was the beta. Maduro proved the model the morning we took delivery of him. The President called it "working out brilliantly for both." He's right. It works brilliantly for both of us, and the customer will come around to the "both" on their own time. Kharg Island is the rollout. A third the size of Manhattan, and 90 percent of Iran's crude runs through it. 2 million barrels a day, one valve. I have prayed my whole career for a country that legible. 53 billion in revenue. 11 percent of their GDP. All of it funneling through a rock you could walk across before lunch. I kept the Venezuela playbook. I changed the classification banner and let find-and-replace do Caracas to Tehran. That was the plan. That was the whole plan. Page 14 still says Caracas. No one on the distribution list has ever reached page 14. They initial page 1. Page 1 is projected revenue. It goes up and to the right, the only direction this town can read. The Energy Secretary testified he was "unaware." Under oath, to Congress, the man who runs American energy. Good. He was never read into the compartment. The fewer names on the account, the cleaner the account. He runs the department. He does not run this. We don't say invasion. We say onboarding. Every onboarding has friction. The Marines off the coast are not an occupying force. They're field engagement. Iran said an invasion of its islands would "shatter all restraint." I logged it as customer feedback and filed it. The file is full. Engaged customers submit the most feedback. It's the quiet ones who churn. And Iran can't churn. That's the part I'm proudest of. We built the account with no exit. No competitor to switch to. We are the market, and the market is now total. Total is my word. I workshopped it. "Seize" scares people. "Total control of their markets" tests at 71, because nobody pictures a kid from Ohio on a beach when you say markets. We moved 22 tankers through Hormuz at night with the lights off. The President said he wanted to announce it so badly it hurt, and he held it a month anyway. That is the most discipline this administration has ever shown. 100 million barrels, he said. The fact-checkers say the math doesn't hold. They're junior. The number was never the deliverable. The number is the soft launch. He said he loves the inflation. People clutched their receipts. 90 a barrel instead of 250 is the discount, and 250 is a price I invented so this one would feel like mercy. How long will we be there? The President answered that. "We had to be there for a while." A while is my favorite contract length. It renews on its own. No court can find the end date. He wondered out loud whether America has the stomach for it. I don't need the country's stomach. I need its signature on page 1. He called the people who object stupid. I'd defend them. They're not stupid. They just read to page 14. Last thing. Annex C. The out-year targets. 3 more, redacted, marked pre-decisional. I can't tell you which 3. Total control of their markets. Nobody in that room asked which market. The crude still sails east, to China, the same as yesterday. We don't own the oil. We own the valve, the fear, and the word "total." That was always the only product.

Peter Girnus 🦅

22,034 views • 3 months ago

a moonshot engineer leaked the benchmark anthropic, openai and xai all buried the same week: kimi k3 beat opus 5, gpt-5.6 and grok 4.6 at $0.94 a task. stop paying anthropic $200 a month for opus 5 and openai $200 for gpt-5.6 when kimi does the same work for $8 the leak showed kimi k3 winning 9 of 12 categories against opus 5, gpt-5.6 and grok 4.6. within 48 hours all three labs quietly pushed pricing pages and one very specific comparison chart off their sites. nobody announced anything. they just deleted, which tells you everything the four numbers they scrubbed: cost per task · $0.94 vs $1.80 -> opus 5 charges $1.80 to finish one task. gpt-5.6 $1.04. grok 4.6 $0.61. kimi k3 $0.94 and it landed 487 of 500 clean -> anthropic is billing you double for a model that lost the benchmark it paid to promote the weights · free, sitting on huggingface right now -> the entire model is a public download. pull it, keep it, run it forever, nobody can switch it off -> a model you can hold cannot be rented at $200 a month. that single fact is what three labs deleted a chart over the switch · one line of bash -> moonshot ships an anthropic-compatible endpoint. one env variable and claude code points at kimi -> same cli, same keybindings, same /model. you change a url, opus 5 never knows it lost the seat the bill · $400 down to $8 -> opus 5 max plus gpt-5.6 pro is $400 a month. kimi runs the same daily work for $8 metered -> that is a 98% cut for output that beat both of them 9 categories to 3 here is the part they will fight me on: the frontier tax died the week this leaked and all three labs know it. once the weights are public the price has a ceiling, because anyone can serve the same model. anthropic, openai and xai are charging 2025 prices on a lead that ended in a benchmark they deleted instead of answered drop your $400/mo ai stack to $8. the run above is kimi k3 finishing the task opus 5 bills $1.80 for. the full breakdown is in the article below
0:19

a moonshot engineer leaked the benchmark anthropic, openai and xai all buried the same week: kimi k3 beat opus 5, gpt-5.6 and grok 4.6 at $0.94 a task. stop paying anthropic $200 a month for opus 5 and openai $200 for gpt-5.6 when kimi does the same work for $8 the leak showed kimi k3 winning 9 of 12 categories against opus 5, gpt-5.6 and grok 4.6. within 48 hours all three labs quietly pushed pricing pages and one very specific comparison chart off their sites. nobody announced anything. they just deleted, which tells you everything the four numbers they scrubbed: cost per task · $0.94 vs $1.80 -> opus 5 charges $1.80 to finish one task. gpt-5.6 $1.04. grok 4.6 $0.61. kimi k3 $0.94 and it landed 487 of 500 clean -> anthropic is billing you double for a model that lost the benchmark it paid to promote the weights · free, sitting on huggingface right now -> the entire model is a public download. pull it, keep it, run it forever, nobody can switch it off -> a model you can hold cannot be rented at $200 a month. that single fact is what three labs deleted a chart over the switch · one line of bash -> moonshot ships an anthropic-compatible endpoint. one env variable and claude code points at kimi -> same cli, same keybindings, same /model. you change a url, opus 5 never knows it lost the seat the bill · $400 down to $8 -> opus 5 max plus gpt-5.6 pro is $400 a month. kimi runs the same daily work for $8 metered -> that is a 98% cut for output that beat both of them 9 categories to 3 here is the part they will fight me on: the frontier tax died the week this leaked and all three labs know it. once the weights are public the price has a ceiling, because anyone can serve the same model. anthropic, openai and xai are charging 2025 prices on a lead that ended in a benchmark they deleted instead of answered drop your $400/mo ai stack to $8. the run above is kimi k3 finishing the task opus 5 bills $1.80 for. the full breakdown is in the article below

starmex

32,547 views • 21 days ago

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.
0:18

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Akshay 🚀

244,990 views • 2 months ago

a contractor in Shenzhen priced a ¥12,470,900 hospital contract, about $1.7m, in one afternoon and beat firms carrying forty people he explained how he did it: the bid consultancy he used to pay took three days and ¥46,000 for the same envelope. he did this one alone, off one screen, at 11.4% margin, uploaded before the 17:00 cutoff 214 pages of tender documents read, 68 binding clauses pulled out, 9,485 building parts loaded, 14 places found where a duct and a beam sit in the same cubic metre, deepest one 38mm, all of them fixed, 3,318 lines of quantities priced and the package encrypted and uploaded before the 17:00 cutoff this is Graph Engineering: the job gets cut into small nodes, one narrow task each, wired so that one node's output is the next node's input, and any node is allowed to stop the whole run. it turns a model that answers you into a machine that finishes the job: - give every node one job and one output. a node doing two things fails at both and you cannot tell which one broke - put the cheapest rejection first. his qualification node reads clause 7.4, foreign-owned firms barred, and ends the run four seconds in, before anything expensive touches the model - what moves between nodes is a file. the model travels as a model, the quantities as a table, the price as a number - build exactly one loop: the checker finds 14 collisions, the fixer drops the duct 550mm, the checker runs again, and nothing moves on until the count is zero - cap that loop, or a graph will grind on three impossible clashes until the deadline passes - keep one node whose only job is to say no, and give it authority over everything above it - log each node's output on its own, because when the price comes out wrong you need to know which node believed the wrong thing - run the expensive nodes last, always the catch is that a graph is an extremely confident machine: point it at an outdated rate book and it prices an entire hospital off it without a single node noticing, because no node is asked to doubt the input, only to process it so the nodes that earn their keep are the ones that reject, and almost nobody builds those first bookmark this, the full build with all nine nodes and what each one hands to the next is written out in the article ↓
0:16

a contractor in Shenzhen priced a ¥12,470,900 hospital contract, about $1.7m, in one afternoon and beat firms carrying forty people he explained how he did it: the bid consultancy he used to pay took three days and ¥46,000 for the same envelope. he did this one alone, off one screen, at 11.4% margin, uploaded before the 17:00 cutoff 214 pages of tender documents read, 68 binding clauses pulled out, 9,485 building parts loaded, 14 places found where a duct and a beam sit in the same cubic metre, deepest one 38mm, all of them fixed, 3,318 lines of quantities priced and the package encrypted and uploaded before the 17:00 cutoff this is Graph Engineering: the job gets cut into small nodes, one narrow task each, wired so that one node's output is the next node's input, and any node is allowed to stop the whole run. it turns a model that answers you into a machine that finishes the job: - give every node one job and one output. a node doing two things fails at both and you cannot tell which one broke - put the cheapest rejection first. his qualification node reads clause 7.4, foreign-owned firms barred, and ends the run four seconds in, before anything expensive touches the model - what moves between nodes is a file. the model travels as a model, the quantities as a table, the price as a number - build exactly one loop: the checker finds 14 collisions, the fixer drops the duct 550mm, the checker runs again, and nothing moves on until the count is zero - cap that loop, or a graph will grind on three impossible clashes until the deadline passes - keep one node whose only job is to say no, and give it authority over everything above it - log each node's output on its own, because when the price comes out wrong you need to know which node believed the wrong thing - run the expensive nodes last, always the catch is that a graph is an extremely confident machine: point it at an outdated rate book and it prices an entire hospital off it without a single node noticing, because no node is asked to doubt the input, only to process it so the nodes that earn their keep are the ones that reject, and almost nobody builds those first bookmark this, the full build with all nine nodes and what each one hands to the next is written out in the article ↓

Argona

38,189 views • 1 month ago

THAT $70 "RUN YOUR OWN LLMS" PI KIT CAN'T RUN A SINGLE LLM. IT'S A VISION CHIP WITH NO RAM. that clip sells a raspberry pi 5 in a slick case with an ai accelerator and the caption "your own llms." clean build, fun kit. the claim is where it breaks. the fine print: the popular $70 pi ai kit uses a hailo-8l, 13 tops. it's built for vision, object detection and image processing, and it has no memory of its own. so it cannot run large language models. full stop the board that actually can is a different one: the newer ai hat+ 2, hailo-10h, 40 tops, with 8gb of dedicated ram. that's $130, not $70 and even that runs only tiny models. llama 3.2 at 1b, qwen 2.5 at 1.5b, deepseek r1 at 1.5b. edge llms live in the 1-7b range, against cloud models at 500b to 2 trillion so the honest pitch: for $130 you can run a very small language model on a pi, slowly, as a fun learning project. that's real and it's cool. "your own llms" on a $70 vision kit is not. why this keeps happening: "ai kit" and a big "tops" number sell. tops sounds like intelligence. but tops measures vision-style math, not whether the chip has the memory to hold a language model. the spec that matters for llms is ram, and the cheap kit has none. the honest caveats, both ways: the $70 kit is genuinely great, just at vision. cameras, object detection, that's its job the $130 hat really does run small llms locally, which a pi couldn't do at all two years ago. that's progress "small" is the load-bearing word. don't expect gpt at home on a pi the takeaway: before you buy a kit because the caption says llm, check two numbers. not the tops. the ram, and the size of the model it can actually load. no 70-dollar miracle, no gpt in a pi case, no tops number that means what you think. save this before you buy the wrong kit for the word on the box.
0:17

THAT $70 "RUN YOUR OWN LLMS" PI KIT CAN'T RUN A SINGLE LLM. IT'S A VISION CHIP WITH NO RAM. that clip sells a raspberry pi 5 in a slick case with an ai accelerator and the caption "your own llms." clean build, fun kit. the claim is where it breaks. the fine print: the popular $70 pi ai kit uses a hailo-8l, 13 tops. it's built for vision, object detection and image processing, and it has no memory of its own. so it cannot run large language models. full stop the board that actually can is a different one: the newer ai hat+ 2, hailo-10h, 40 tops, with 8gb of dedicated ram. that's $130, not $70 and even that runs only tiny models. llama 3.2 at 1b, qwen 2.5 at 1.5b, deepseek r1 at 1.5b. edge llms live in the 1-7b range, against cloud models at 500b to 2 trillion so the honest pitch: for $130 you can run a very small language model on a pi, slowly, as a fun learning project. that's real and it's cool. "your own llms" on a $70 vision kit is not. why this keeps happening: "ai kit" and a big "tops" number sell. tops sounds like intelligence. but tops measures vision-style math, not whether the chip has the memory to hold a language model. the spec that matters for llms is ram, and the cheap kit has none. the honest caveats, both ways: the $70 kit is genuinely great, just at vision. cameras, object detection, that's its job the $130 hat really does run small llms locally, which a pi couldn't do at all two years ago. that's progress "small" is the load-bearing word. don't expect gpt at home on a pi the takeaway: before you buy a kit because the caption says llm, check two numbers. not the tops. the ram, and the size of the model it can actually load. no 70-dollar miracle, no gpt in a pi case, no tops number that means what you think. save this before you buy the wrong kit for the word on the box.

RetroChainer

11,100 views • 1 month ago

If you have an RTX 3090 or 4090, Mia just shipped you a free massive upgrade in both speed and intelligence. I will explain to you why this will make your Qwen 3.8 27B on your card, even better, and my flags for running it. Qwen3.8-27B, EXL3 3.5bpw, DFlash2 speculative decode, RTX 4090. Single stream. The kit is from MiaAI-Lab, EXL3 is turboderp's format. I re-measured everything on my own card because the my first benchmarks seemed off. It turns out it really does run much faster. WHY EXL3 IS A DIFFERENT ANIMAL The old way (Q4_K_M) rounds each weight to the nearest 4-bit value independently. Every weight introduces its own rounding error. Those errors accumulate across millions of weights and causes drift (Slightly dumber). EXL3 is a fundamentally different compression algorithm. Instead of rounding each weight on its own, it encodes the entire weight vector as a path through a constrained codebook and spreads the rounding error across dimensions using a Hadamard transform. The result is that at the same bits per weight, more of the original model's intelligence is preserved. The important part is this CAN ACTUALLY BE MEASURED. The cleanest way to see that is KL divergence against a high-precision teacher. Lower means the quantized model thinks more like the original. On the malaiwah independent teacher-logit panel for GLM-5.3-Flash: EXL3 4bpw: 0.0246 nats Official FP8: 0.0206 nats NVFP4: 0.0605 nats EXL3 sits 0.004 nats behind native FP8 at half the size. NVFP4 at higher bit width is 2.5x further from the teacher. That panel is GLM-5.3-Flash, not Qwen 3.8. Cited as the mechanism, not as this run's data. But the point stands: EXL3 is not just smaller, it is smarter per bit than the formats most people are running. WHAT I MEASURED I first measure 108 tok/s from a single run. After that number looked too good to be true. I reran it. It looks like after a warm up, the numbers are even better. Basically, like people long thought, the RTX 3090 and RTX 4090 are actually superb AI computer cards. Hence why NVIDIA stopped shipping them with NVLINK since the 4090. Short context ceiling (~2k in, 1016-token output, TTFT-separated): 135, 138, 153, 174, 133, 129 tok/s across 6 runs. Sustained longform (2040-token essay): 105.4, 94.5, 98.4 tok/s Short answer (504 tokens): 93.2 tok/s The honest shape: ~130-150 tok/s at short context is the ceiling, ~94-105 sustained on longform. The ceiling matters because that is what people feel in chat. The old dense Q4_K_M on llama.cpp ran ~37 tok/s on this same card. (No MTP), with MTP about 60 tok/s Sustained is roughly 2.5-3x. Ceiling is closer to 4x. Same model, different quantization and engine. The multiplier comes from EXL3, the ExLlamaV2 engine, and DFlash2 together. CONCURRENCY IS A RTX 4090 LANE. Just like the old config on the 4090, the 24gb vram, means a long context can only hold one stream, and running concurrency requires to lower context length, because it runs fast it sort of makes up for it by being faster than slower GPU chips. CONTEXT LADDER The recipe doc measured prefill. I re-ran it with TTFT separated from decode, because decode is what you actually feel after the first token. ~5k in: decode 140 tok/s (TTFT 0.5s), needle HIT ~18k in: decode 85 tok/s (TTFT 0.3s*), needle HIT ~73k in: decode 28 tok/s (TTFT 1.8s), needle HIT ~146k in: decode 16 tok/s (TTFT 2.3s), needle HIT (*0.3s at 18k is a prefix-cache hit from the paired pass. Cold prefill for reference: ~2,020 tok/s at 17k falling to ~508 at 153k.) Needle hit at every depth, mine and the original 7/7. Retrieval is intact at max context. Speed is not: decode falls ~9x from short to max. Past ~50k tokens this stops being a chat tool and becomes a batch tool. At 146k it works, but nobody is typing interactively against 16 tok/s. WHERE IT BROKE The model's native context is 262k. The README says DFlash2 fits ~220k on a 24GB card. My 200,704-token attempt failed with insufficient VRAM. Dropped to 168,960 and it booted. The real ceiling is somewhere between 168,960 and 200,704 and I never tested that gap. I jumped to a value that worked and called it done. That is ~32k tokens of context I left on the table. One thing the numbers taught me: JSON tokenizes at ~1.5 chars/token, prose at ~3.9. "150k tokens of JSON" needs ~2.7x more filler than the same estimate in prose. Size by real tokens, not estimates. THE UPGRADE If you own a 4090 and you are running Q4_K_M on llama.cpp, you are leaving a good bit of speed and measurable intelligence on the table. The same model, on the same card, with a better quantization and engine, goes from 60 tok/s to 130-150 at short context and 94-105 sustained. The model also thinks closer to the original because EXL3 preserves more of the output distribution per bit than the old rounding method. The recipe is in the first reply. Everything above came from one 4090 and one afternoon of re-measuring. The decode ladder especially needs independent numbers. If your card gets different falloff, that is worth knowing. Recipe and flags/ findings in reply 👇
0:15

If you have an RTX 3090 or 4090, Mia just shipped you a free massive upgrade in both speed and intelligence. I will explain to you why this will make your Qwen 3.8 27B on your card, even better, and my flags for running it. Qwen3.8-27B, EXL3 3.5bpw, DFlash2 speculative decode, RTX 4090. Single stream. The kit is from MiaAI-Lab, EXL3 is turboderp's format. I re-measured everything on my own card because the my first benchmarks seemed off. It turns out it really does run much faster. WHY EXL3 IS A DIFFERENT ANIMAL The old way (Q4_K_M) rounds each weight to the nearest 4-bit value independently. Every weight introduces its own rounding error. Those errors accumulate across millions of weights and causes drift (Slightly dumber). EXL3 is a fundamentally different compression algorithm. Instead of rounding each weight on its own, it encodes the entire weight vector as a path through a constrained codebook and spreads the rounding error across dimensions using a Hadamard transform. The result is that at the same bits per weight, more of the original model's intelligence is preserved. The important part is this CAN ACTUALLY BE MEASURED. The cleanest way to see that is KL divergence against a high-precision teacher. Lower means the quantized model thinks more like the original. On the malaiwah independent teacher-logit panel for GLM-5.3-Flash: EXL3 4bpw: 0.0246 nats Official FP8: 0.0206 nats NVFP4: 0.0605 nats EXL3 sits 0.004 nats behind native FP8 at half the size. NVFP4 at higher bit width is 2.5x further from the teacher. That panel is GLM-5.3-Flash, not Qwen 3.8. Cited as the mechanism, not as this run's data. But the point stands: EXL3 is not just smaller, it is smarter per bit than the formats most people are running. WHAT I MEASURED I first measure 108 tok/s from a single run. After that number looked too good to be true. I reran it. It looks like after a warm up, the numbers are even better. Basically, like people long thought, the RTX 3090 and RTX 4090 are actually superb AI computer cards. Hence why NVIDIA stopped shipping them with NVLINK since the 4090. Short context ceiling (~2k in, 1016-token output, TTFT-separated): 135, 138, 153, 174, 133, 129 tok/s across 6 runs. Sustained longform (2040-token essay): 105.4, 94.5, 98.4 tok/s Short answer (504 tokens): 93.2 tok/s The honest shape: ~130-150 tok/s at short context is the ceiling, ~94-105 sustained on longform. The ceiling matters because that is what people feel in chat. The old dense Q4_K_M on llama.cpp ran ~37 tok/s on this same card. (No MTP), with MTP about 60 tok/s Sustained is roughly 2.5-3x. Ceiling is closer to 4x. Same model, different quantization and engine. The multiplier comes from EXL3, the ExLlamaV2 engine, and DFlash2 together. CONCURRENCY IS A RTX 4090 LANE. Just like the old config on the 4090, the 24gb vram, means a long context can only hold one stream, and running concurrency requires to lower context length, because it runs fast it sort of makes up for it by being faster than slower GPU chips. CONTEXT LADDER The recipe doc measured prefill. I re-ran it with TTFT separated from decode, because decode is what you actually feel after the first token. ~5k in: decode 140 tok/s (TTFT 0.5s), needle HIT ~18k in: decode 85 tok/s (TTFT 0.3s*), needle HIT ~73k in: decode 28 tok/s (TTFT 1.8s), needle HIT ~146k in: decode 16 tok/s (TTFT 2.3s), needle HIT (*0.3s at 18k is a prefix-cache hit from the paired pass. Cold prefill for reference: ~2,020 tok/s at 17k falling to ~508 at 153k.) Needle hit at every depth, mine and the original 7/7. Retrieval is intact at max context. Speed is not: decode falls ~9x from short to max. Past ~50k tokens this stops being a chat tool and becomes a batch tool. At 146k it works, but nobody is typing interactively against 16 tok/s. WHERE IT BROKE The model's native context is 262k. The README says DFlash2 fits ~220k on a 24GB card. My 200,704-token attempt failed with insufficient VRAM. Dropped to 168,960 and it booted. The real ceiling is somewhere between 168,960 and 200,704 and I never tested that gap. I jumped to a value that worked and called it done. That is ~32k tokens of context I left on the table. One thing the numbers taught me: JSON tokenizes at ~1.5 chars/token, prose at ~3.9. "150k tokens of JSON" needs ~2.7x more filler than the same estimate in prose. Size by real tokens, not estimates. THE UPGRADE If you own a 4090 and you are running Q4_K_M on llama.cpp, you are leaving a good bit of speed and measurable intelligence on the table. The same model, on the same card, with a better quantization and engine, goes from 60 tok/s to 130-150 at short context and 94-105 sustained. The model also thinks closer to the original because EXL3 preserves more of the output distribution per bit than the old rounding method. The recipe is in the first reply. Everything above came from one 4090 and one afternoon of re-measuring. The decode ladder especially needs independent numbers. If your card gets different falloff, that is worth knowing. Recipe and flags/ findings in reply 👇

Yume_X

38,079 views • 8 days ago

A 16-year-old in Austin made $49,200 in six months while every law firm in his city was busy counting Google reviews that nobody under 30 reads anymore. He walked into a law firm and asked the paralegal to search for the practice on Perplexity. The paralegal laughed and pointed at 400 five-star reviews on Google. He said, "Just do it." Perplexity had never heard of them. Here is what the kid understood that the paralegal did not. Google reviews are a ranking signal inside Google's algorithm. Perplexity runs its own crawler. It does not care how many stars you have on a platform it is not reading. It pulls from legal directories, bar association profiles, Yelp, structured schema data, and third-party citations. A firm can sit at the top of the Google Local Pack with 847 reviews and have zero citation presence inside the AI systems that 500 million users query every month. As of February 2026, the overlap between pages ranking in Google's top 10 and pages cited inside AI-generated answers had collapsed from 76 percent to under 20 percent. Two entirely different systems. Almost nobody in legal had noticed. The paralegal thought the reviews were the proof. The kid saw they were the blind spot. So he built a $1,200 audit. The deliverable is a single document. He opens Perplexity, ChatGPT, and Claude. He types the firm's practice area and city. He screenshots what comes back. Then he runs the same search on every competitor in the market. He maps which firms are being named, where the citations are coming from, and what data signals are missing from the ones that do not appear. The finding is almost always identical: No Foursquare listing No attorney schema or LegalService markup beyond the default WordPress install Bar association profile unlinked from the main site Attorney bios with no verifiable credentials structured for machine reading NAP inconsistent across the seven directories that Perplexity actually indexes A firm charging $450 an hour that ChatGPT cannot confidently recommend because it cannot verify the address matches across three platforms. Less than 5 percent of local businesses have done this work as of 2026. In legal, the number is closer to zero. He charges $1,200 to show them exactly where they do not exist. Then he quotes them the fix. He walked out of the first firm with a check. That firm referred him to two others before the week was over. Those two referred three more. He has never made a cold call. He has never run an ad. He does not have a website. 41 firms in six months. $49,200 in revenue. He is 16. From what I have observed, the arbitrage here is not technical. It is perceptual. Law firms spent a decade optimizing for a system that is no longer the first place their clients look. The 16-year-old simply walked in and showed them the new one.
0:15

A 16-year-old in Austin made $49,200 in six months while every law firm in his city was busy counting Google reviews that nobody under 30 reads anymore. He walked into a law firm and asked the paralegal to search for the practice on Perplexity. The paralegal laughed and pointed at 400 five-star reviews on Google. He said, "Just do it." Perplexity had never heard of them. Here is what the kid understood that the paralegal did not. Google reviews are a ranking signal inside Google's algorithm. Perplexity runs its own crawler. It does not care how many stars you have on a platform it is not reading. It pulls from legal directories, bar association profiles, Yelp, structured schema data, and third-party citations. A firm can sit at the top of the Google Local Pack with 847 reviews and have zero citation presence inside the AI systems that 500 million users query every month. As of February 2026, the overlap between pages ranking in Google's top 10 and pages cited inside AI-generated answers had collapsed from 76 percent to under 20 percent. Two entirely different systems. Almost nobody in legal had noticed. The paralegal thought the reviews were the proof. The kid saw they were the blind spot. So he built a $1,200 audit. The deliverable is a single document. He opens Perplexity, ChatGPT, and Claude. He types the firm's practice area and city. He screenshots what comes back. Then he runs the same search on every competitor in the market. He maps which firms are being named, where the citations are coming from, and what data signals are missing from the ones that do not appear. The finding is almost always identical: No Foursquare listing No attorney schema or LegalService markup beyond the default WordPress install Bar association profile unlinked from the main site Attorney bios with no verifiable credentials structured for machine reading NAP inconsistent across the seven directories that Perplexity actually indexes A firm charging $450 an hour that ChatGPT cannot confidently recommend because it cannot verify the address matches across three platforms. Less than 5 percent of local businesses have done this work as of 2026. In legal, the number is closer to zero. He charges $1,200 to show them exactly where they do not exist. Then he quotes them the fix. He walked out of the first firm with a check. That firm referred him to two others before the week was over. Those two referred three more. He has never made a cold call. He has never run an ad. He does not have a website. 41 firms in six months. $49,200 in revenue. He is 16. From what I have observed, the arbitrage here is not technical. It is perceptual. Law firms spent a decade optimizing for a system that is no longer the first place their clients look. The 16-year-old simply walked in and showed them the new one.

Argona

478,087 views • 4 months ago

TIMED DIALOGUE IN A NIGHTCLUB. THREE WALLS FALL AT ONCE. Nightclub sketch, cut in two halves. Black-and-white first - a couple making out on a couch, someone laughing off-camera. Then color reveals the setup: guy walks up with a drink, delivers a line, she gives him a one-sentence answer that changes the picture, he pauses, then kisses her anyway. None of them exist. It's fully generated, both halves. - What used to be four problems is now one clip Character consistency across a cut - same two faces in B&W and in color. Two-person dialogue with alternating lip sync - three separate English lines, all on time, all matching mouth shapes. Nightclub lighting - low light, saturated color wash, moving sources - was the last hard lighting environment for AI video to render without collapsing into noise. And a kiss - two faces contacting without merging into each other, which has been one of the persistent tells. Any one of these has been solvable for maybe six months. All four in one sketch was still a demo-reel problem in early 2026. - The B&W cut is doing two jobs The editing choice isn't style. It's engineering. Splitting a 15-second sketch into two 5-7 second clips means the model only has to hold consistency inside each segment, not across the whole thing. Monochrome also hides small differences between the two generations - if the girl's face is 3% off between the halves, B&W flattens the delta. Color grading in the second half does the reverse job. Two seams, both hidden by the aesthetic. - The comic beat is the actual craft Generating a kiss is one problem. Generating a kiss that lands as a punchline is a different one. The half-second where he pauses, processes, and decides not to care - that timing has to be prompted specifically. The default output of every current model is a rushed sequence with no beats. Deadpan comic delivery out of AI video means the operator wrote the prompt the way a screenwriter would - pauses, reactions, holds, all specified frame by frame. - What it costs Two 5-7 second clips at $3-5 each with in-model audio. Locked character references for both actors so the faces match across the cut. Prompt structured as a mini-script with beat notation. Realistically 40-60 rerolls to land the timing on all three spoken lines and the kiss. Under $200 in compute. A weekend from concept to publish-ready. - What this actually opens Short-form comedy has been the one segment of content nobody was making with AI video yet, because you can't fake comic timing when your output has drift and glitches. That barrier just came down. Which means every sketch account, every meme page, every stand-up clip factory now has a pipeline that doesn't require booking actors, renting a location, or getting a laugh out of a live crew. That's a real shift in a market that produces billions of views a month.
0:16

TIMED DIALOGUE IN A NIGHTCLUB. THREE WALLS FALL AT ONCE. Nightclub sketch, cut in two halves. Black-and-white first - a couple making out on a couch, someone laughing off-camera. Then color reveals the setup: guy walks up with a drink, delivers a line, she gives him a one-sentence answer that changes the picture, he pauses, then kisses her anyway. None of them exist. It's fully generated, both halves. - What used to be four problems is now one clip Character consistency across a cut - same two faces in B&W and in color. Two-person dialogue with alternating lip sync - three separate English lines, all on time, all matching mouth shapes. Nightclub lighting - low light, saturated color wash, moving sources - was the last hard lighting environment for AI video to render without collapsing into noise. And a kiss - two faces contacting without merging into each other, which has been one of the persistent tells. Any one of these has been solvable for maybe six months. All four in one sketch was still a demo-reel problem in early 2026. - The B&W cut is doing two jobs The editing choice isn't style. It's engineering. Splitting a 15-second sketch into two 5-7 second clips means the model only has to hold consistency inside each segment, not across the whole thing. Monochrome also hides small differences between the two generations - if the girl's face is 3% off between the halves, B&W flattens the delta. Color grading in the second half does the reverse job. Two seams, both hidden by the aesthetic. - The comic beat is the actual craft Generating a kiss is one problem. Generating a kiss that lands as a punchline is a different one. The half-second where he pauses, processes, and decides not to care - that timing has to be prompted specifically. The default output of every current model is a rushed sequence with no beats. Deadpan comic delivery out of AI video means the operator wrote the prompt the way a screenwriter would - pauses, reactions, holds, all specified frame by frame. - What it costs Two 5-7 second clips at $3-5 each with in-model audio. Locked character references for both actors so the faces match across the cut. Prompt structured as a mini-script with beat notation. Realistically 40-60 rerolls to land the timing on all three spoken lines and the kiss. Under $200 in compute. A weekend from concept to publish-ready. - What this actually opens Short-form comedy has been the one segment of content nobody was making with AI video yet, because you can't fake comic timing when your output has drift and glitches. That barrier just came down. Which means every sketch account, every meme page, every stand-up clip factory now has a pipeline that doesn't require booking actors, renting a location, or getting a laugh out of a live crew. That's a real shift in a market that produces billions of views a month.

capONE 💎

82,533 views • 27 days ago

I went a little overboard with Codex last week and burned through my entire weekly allowance in two days. Luckily, my quota reset today. Otherwise, I’m not sure what I would’ve done. It got me thinking: instead of asking one large model to handle everything from start to finish, why not let a stronger model plan the project and review the work, while a model built for execution handles the day-to-day implementation? So I tried it. The result was better than I expected. I used GPT-5.6 Sol in Codex as the decision-maker, then ran Ling-3.0-flash from Ant Ling inside OpenCode as the execution engine. Together, they built a small 3D farming game. Before writing any code, I had Codex create four documents: SPEC.md defined the product scope and the lines we couldn’t cross. ARCHITECTURE.md laid out the isometric coordinate system, state machine, and module boundaries. TASKS.md broke the project into small jobs Ling could tackle one at a time. ACCEPTANCE.md explained how each step would be tested and what “done” actually meant. Then I gave Ling a very straightforward role: You are the execution model for this project. Read all four documents before you begin. Work only on the task assigned for this round. When you’re done, run typecheck, test, and build. If anything fails, read the error, fix it, and run the checks again. Do not move on to the next task early. Ling handled dependency installation, project structure, strict TypeScript configuration, test setup, and a production build in 6 minutes and 3 seconds. It ran into issues with the Vite test config, a TS6310 error, and a missing jsdom dependency along the way. Instead of stopping at the first error, it kept reading the logs and fixing the problems until all three checks passed. The speed was honestly hard to believe. If you exclude the time spent waiting on tools, it was producing more than 100 tokens per second. That made the whole development loop feel noticeably faster. After this experiment, I’m planning to keep using the same workflow. If the task is small, there’s no reason to call an expensive planning model for every single step. If the task is large, handing the entire project to a Flash model in one prompt isn’t a great idea either. The setup that makes more sense to me is: Use a more capable model such as Codex to explore the project, make architectural decisions, and break the work down. Put the constraints into specs, schemas, types, and tests instead of leaving them buried in chat history. Give Ling-3.0-flash a steady stream of clear, verifiable implementation tasks. Report bugs with structured context and actual error logs, rather than saying, “It still doesn’t work.” Bring Codex back in for architecture reviews, visual checks, and changes that affect multiple parts of the project. The point of this setup isn’t to give AI a big “build the whole project” button. It’s to turn software development into a pipeline with a much more sensible cost structure: Codex figures out the plan, sets the boundaries, and catches problems. Ling-3.0-flash moves quickly, calls tools reliably, and works through well-defined tasks at scale. For agent workflows that involve lots of repetitive edits, production tasks, and tool calls, this may be a more practical answer than simply using the biggest model for everything.
0:16

I went a little overboard with Codex last week and burned through my entire weekly allowance in two days. Luckily, my quota reset today. Otherwise, I’m not sure what I would’ve done. It got me thinking: instead of asking one large model to handle everything from start to finish, why not let a stronger model plan the project and review the work, while a model built for execution handles the day-to-day implementation? So I tried it. The result was better than I expected. I used GPT-5.6 Sol in Codex as the decision-maker, then ran Ling-3.0-flash from Ant Ling inside OpenCode as the execution engine. Together, they built a small 3D farming game. Before writing any code, I had Codex create four documents: SPEC.md defined the product scope and the lines we couldn’t cross. ARCHITECTURE.md laid out the isometric coordinate system, state machine, and module boundaries. TASKS.md broke the project into small jobs Ling could tackle one at a time. ACCEPTANCE.md explained how each step would be tested and what “done” actually meant. Then I gave Ling a very straightforward role: You are the execution model for this project. Read all four documents before you begin. Work only on the task assigned for this round. When you’re done, run typecheck, test, and build. If anything fails, read the error, fix it, and run the checks again. Do not move on to the next task early. Ling handled dependency installation, project structure, strict TypeScript configuration, test setup, and a production build in 6 minutes and 3 seconds. It ran into issues with the Vite test config, a TS6310 error, and a missing jsdom dependency along the way. Instead of stopping at the first error, it kept reading the logs and fixing the problems until all three checks passed. The speed was honestly hard to believe. If you exclude the time spent waiting on tools, it was producing more than 100 tokens per second. That made the whole development loop feel noticeably faster. After this experiment, I’m planning to keep using the same workflow. If the task is small, there’s no reason to call an expensive planning model for every single step. If the task is large, handing the entire project to a Flash model in one prompt isn’t a great idea either. The setup that makes more sense to me is: Use a more capable model such as Codex to explore the project, make architectural decisions, and break the work down. Put the constraints into specs, schemas, types, and tests instead of leaving them buried in chat history. Give Ling-3.0-flash a steady stream of clear, verifiable implementation tasks. Report bugs with structured context and actual error logs, rather than saying, “It still doesn’t work.” Bring Codex back in for architecture reviews, visual checks, and changes that affect multiple parts of the project. The point of this setup isn’t to give AI a big “build the whole project” button. It’s to turn software development into a pipeline with a much more sensible cost structure: Codex figures out the plan, sets the boundaries, and catches problems. Ling-3.0-flash moves quickly, calls tools reliably, and works through well-defined tasks at scale. For agent workflows that involve lots of repetitive edits, production tasks, and tool calls, this may be a more practical answer than simply using the biggest model for everything.

雪踏乌云

23,107 views • 1 month ago

I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.
0:12

I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.

Alok

170,442 views • 2 months ago

HERMES AGENT HAS A SECOND BRAIN. 1,100+ KNOWLEDGE FILES. AUTO-LINKED. SELF-IMPROVING. GROWING EVERY NIGHT. THIS IS THE OBSIDIAN GRAPH BEHIND IT. every dot = one knowledge file (markdown) every line = one wiki-link between files every color = one category (skills, notes, decisions, sources, entities) HOW IT BUILDS ITSELF: Hermes ships with a bundled LLM Wiki skill. based on Andrej Karpathy's pattern. unlike RAG (rediscovers knowledge from scratch every query), the wiki compiles knowledge once and keeps it current. when you feed the agent a source: → it reads the content → writes a structured markdown page → auto-links to every related existing page → flags contradictions with previous entries → updates all affected pages one source in. multiple connections created. the graph grows denser with every entry. WHAT FEEDS THE WIKI: → articles and URLs you find interesting → meeting transcripts → PDF documents and research papers → conversation history from Hermes sessions → Claude Code and Codex session history → Slack logs, email threads, saved notes → YouTube transcripts → raw text dropped into a _raw/ folder the obsidian-wiki package supports multi-agent ingest from Hermes, Claude Code, Codex, OpenClaw, Pi, Windsurf, and ChatGPT exports. install: pip install obsidian-wiki obsidian-wiki setup --vault ~/wiki AUTOMATE THE GROWTH: set cron jobs to feed the wiki overnight: "every day at 9am, check for new meetings. ingest transcripts into the wiki." "every week, check arXiv for new papers in [niche]. summarize and file into the wiki." "every day, ingest today's Hermes sessions into the wiki under session-history." month 1: 50 entries. scattered. month 3: 300+ entries. cross-referenced. month 6: 1,000+ entries. the agent surfaces patterns you never searched for. WHY OBSIDIAN: the wiki is plain markdown files. no database. no lock-in. open it in Obsidian for graph view: → nodes show knowledge density → links show how ideas connect → clusters reveal your strongest domains → orphan nodes reveal gaps Hermes writes from a VPS. Obsidian reads on your laptop. obsidian-headless syncs without a GUI. agent writes from the server, you browse on your device. FOUR MEMORY LAYERS: Layer 1: memory.md + user.md (~2,200 + 1,375 chars. short-term.) Layer 2: SQLite with FTS5 (full session transcripts. searchable.) Layer 3: external providers (Mem0, SuperMemory, Honcho. optional.) Layer 4: Obsidian wiki via LLM Wiki skill (unlimited. compounding. the long-term brain.) layers 1-3 handle memory. layer 4 handles knowledge. the graph in this post is layer 4. SETUP: set in Desktop app, Dashboard, or config.yaml: WIKI_PATH=~/wiki OBSIDIAN_VAULT_PATH=~/wiki first run: Hermes asks for your domain. answer with your niche. the skill builds SCHEMA.md with tag taxonomy. after that: "index this into my wiki: [URL or text]" the wiki grows. the graph densifies. the agent gets smarter because the knowledge base got smarter. full 15 levels breakdown in the article 👇
0:11

HERMES AGENT HAS A SECOND BRAIN. 1,100+ KNOWLEDGE FILES. AUTO-LINKED. SELF-IMPROVING. GROWING EVERY NIGHT. THIS IS THE OBSIDIAN GRAPH BEHIND IT. every dot = one knowledge file (markdown) every line = one wiki-link between files every color = one category (skills, notes, decisions, sources, entities) HOW IT BUILDS ITSELF: Hermes ships with a bundled LLM Wiki skill. based on Andrej Karpathy's pattern. unlike RAG (rediscovers knowledge from scratch every query), the wiki compiles knowledge once and keeps it current. when you feed the agent a source: → it reads the content → writes a structured markdown page → auto-links to every related existing page → flags contradictions with previous entries → updates all affected pages one source in. multiple connections created. the graph grows denser with every entry. WHAT FEEDS THE WIKI: → articles and URLs you find interesting → meeting transcripts → PDF documents and research papers → conversation history from Hermes sessions → Claude Code and Codex session history → Slack logs, email threads, saved notes → YouTube transcripts → raw text dropped into a _raw/ folder the obsidian-wiki package supports multi-agent ingest from Hermes, Claude Code, Codex, OpenClaw, Pi, Windsurf, and ChatGPT exports. install: pip install obsidian-wiki obsidian-wiki setup --vault ~/wiki AUTOMATE THE GROWTH: set cron jobs to feed the wiki overnight: "every day at 9am, check for new meetings. ingest transcripts into the wiki." "every week, check arXiv for new papers in [niche]. summarize and file into the wiki." "every day, ingest today's Hermes sessions into the wiki under session-history." month 1: 50 entries. scattered. month 3: 300+ entries. cross-referenced. month 6: 1,000+ entries. the agent surfaces patterns you never searched for. WHY OBSIDIAN: the wiki is plain markdown files. no database. no lock-in. open it in Obsidian for graph view: → nodes show knowledge density → links show how ideas connect → clusters reveal your strongest domains → orphan nodes reveal gaps Hermes writes from a VPS. Obsidian reads on your laptop. obsidian-headless syncs without a GUI. agent writes from the server, you browse on your device. FOUR MEMORY LAYERS: Layer 1: memory.md + user.md (~2,200 + 1,375 chars. short-term.) Layer 2: SQLite with FTS5 (full session transcripts. searchable.) Layer 3: external providers (Mem0, SuperMemory, Honcho. optional.) Layer 4: Obsidian wiki via LLM Wiki skill (unlimited. compounding. the long-term brain.) layers 1-3 handle memory. layer 4 handles knowledge. the graph in this post is layer 4. SETUP: set in Desktop app, Dashboard, or config.yaml: WIKI_PATH=~/wiki OBSIDIAN_VAULT_PATH=~/wiki first run: Hermes asks for your domain. answer with your niche. the skill builds SCHEMA.md with tag taxonomy. after that: "index this into my wiki: [URL or text]" the wiki grows. the graph densifies. the agent gets smarter because the knowledge base got smarter. full 15 levels breakdown in the article 👇

YanXbt

34,987 views • 2 months ago

i generated over 3 BILLION views and made over $23,000 from one faceless youtube channel beginning of the year before youtube paid me a single cent and the embarrassing part? i did not apply for monetization. until i hit 121k subscribers. that's how little adsense actually mattered to what i built. let me tell you what i was doing instead, the way i'd tell a friend: everyone starting youtube waits for the magic numbers 1,000 subs, 4,000 watch hours before they think they're allowed to make money. that wait is the single biggest mistake on the whole platform. you don't need youtube's permission to make money from youtube. you need three things: a channel getting views (any views), an affiliate offer that fits your audience, and a link in your description. that's it. i ran mine through CPA offers the low friction kind where your viewer doesn't have to buy anything. they sign up, enter a sweepstake, grab a freebie, and you get paid per action. costco, walmart, target, sam's club style stuff. nobody's spending $200. they're claiming something free that they actually wanted anyway. that's why it converts even on a tiny channel. but here's the part everyone gets wrong matching the offer to the niche. you can't slap any offer on any channel. it has to feel like the natural next step from the video they just watched: but here's the part everyone gets wrong matching the offer to the niche. you can't slap any offer on any channel. it has to feel like the natural next step from the video they just watched: budgeting channel? walmart/costco savings offers. they're literally there to save money. giveaway content? sweepstakes offers. the offer IS the content. 55+ money content? membership + gift card offers. older audiences actually complete these. the question i ask before picking anything: "would the person watching this genuinely want it?" if it's not an obvious yes, wrong offer. and the one move almost nobody makes don't send people straight to the raw link. build a simple landing page first. a naked affiliate link looks like spam. a clean little page that says what it is and what to do next looks legit, and it doubles or triples your conversions on the exact same traffic. takes 20 minutes to build now. zero excuse. so my whole machine was: video = link in description = landing page = offer. that quietly made $23k while i wasn't even monetized. the funny part? when i finally turned monetization on at 121k, adsense just stacked on top. now both run at once. but the affiliate side is the part that taught me how to actually sell adsense never teaches you anything. if you're sitting at 300 subs waiting for youtube to pay you, you're waiting for the wrong thing. the description box under your very first video is already a business. i wrote the entire breakdown how i pick offers, build the landing pages, the exact niches that convert in the full thing. it's all here 👇 follow me. i share everything i learned building this the slow, expensive way so you don't have to.
0:12

i generated over 3 BILLION views and made over $23,000 from one faceless youtube channel beginning of the year before youtube paid me a single cent and the embarrassing part? i did not apply for monetization. until i hit 121k subscribers. that's how little adsense actually mattered to what i built. let me tell you what i was doing instead, the way i'd tell a friend: everyone starting youtube waits for the magic numbers 1,000 subs, 4,000 watch hours before they think they're allowed to make money. that wait is the single biggest mistake on the whole platform. you don't need youtube's permission to make money from youtube. you need three things: a channel getting views (any views), an affiliate offer that fits your audience, and a link in your description. that's it. i ran mine through CPA offers the low friction kind where your viewer doesn't have to buy anything. they sign up, enter a sweepstake, grab a freebie, and you get paid per action. costco, walmart, target, sam's club style stuff. nobody's spending $200. they're claiming something free that they actually wanted anyway. that's why it converts even on a tiny channel. but here's the part everyone gets wrong matching the offer to the niche. you can't slap any offer on any channel. it has to feel like the natural next step from the video they just watched: but here's the part everyone gets wrong matching the offer to the niche. you can't slap any offer on any channel. it has to feel like the natural next step from the video they just watched: budgeting channel? walmart/costco savings offers. they're literally there to save money. giveaway content? sweepstakes offers. the offer IS the content. 55+ money content? membership + gift card offers. older audiences actually complete these. the question i ask before picking anything: "would the person watching this genuinely want it?" if it's not an obvious yes, wrong offer. and the one move almost nobody makes don't send people straight to the raw link. build a simple landing page first. a naked affiliate link looks like spam. a clean little page that says what it is and what to do next looks legit, and it doubles or triples your conversions on the exact same traffic. takes 20 minutes to build now. zero excuse. so my whole machine was: video = link in description = landing page = offer. that quietly made $23k while i wasn't even monetized. the funny part? when i finally turned monetization on at 121k, adsense just stacked on top. now both run at once. but the affiliate side is the part that taught me how to actually sell adsense never teaches you anything. if you're sitting at 300 subs waiting for youtube to pay you, you're waiting for the wrong thing. the description box under your very first video is already a business. i wrote the entire breakdown how i pick offers, build the landing pages, the exact niches that convert in the full thing. it's all here 👇 follow me. i share everything i learned building this the slow, expensive way so you don't have to.

Tryahd

22,145 views • 10 days ago