Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Microsoft just a 1-bit LLM with 2B parameters that can run on CPUs like Apple M2. BitNet b1.58 2B4T outperforms fp LLaMA 3.2 1B while using only 0.4GB memory versus 2GB and processes tokens 40% faster. 100% opensource.

Shubham Saboo

113,720 subscribers

260,049 views • 1 year ago •via X (Twitter)

News & Politics Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Microsoft is testing a new feature in Windows 11 called Low Latency Profile that gives a short boost to your CPU for 1 to 3 seconds when you open apps or menus. Early tests show Microsoft apps like Edge and Outlook open up to 40% faster, while the Start menu and other menus can load up to 70% faster, with CPU use jumping high for a few seconds then dropping right back down. Microsoft says this is normal because Apple does it in macOS and Linux, plus phones use the same type of quick CPU boosts for better speed. The feature is only in early Insider testing now, so it is not out for everyone yet.

Pirat_Nation 🔴

145,459 views • 2 months ago

Microsoft's new Outlook takes more than 10 seconds to open an email via notifications, while Outlook Classic opens instantly on Windows 11! The funniest part? If I ignore the notification, open Outlook manually, find the email myself, and click it, I can finish faster than the notification flow. It's all because Outlook for Windows is literally Microsoft Edge running Outlook .com in a browser window. New Outlook runs through WebView2 with around 10 separate processes: GPU process, service worker, utility processes, manager, and more. It uses roughly 490MB to 636MB RAM while idle, compared to Outlook Classic sitting around 117MB to 148MB. CPU usage is also worse: around 4% idle on new Outlook versus under 1% on Classic in my testing. Microsoft shut down Mail and Calendar, keeps pushing enterprises toward this web wrapper, and still wants people to believe it is the future of email on Windows. SHAME!

Microsoft's new Outlook takes more than 10 seconds to open an email via notifications, while Outlook Classic opens instantly on Windows 11! The funniest part? If I ignore the notification, open Outlook manually, find the email myself, and click it, I can finish faster than the notification flow. It's all because Outlook for Windows is literally Microsoft Edge running Outlook .com in a browser window. New Outlook runs through WebView2 with around 10 separate processes: GPU process, service worker, utility processes, manager, and more. It uses roughly 490MB to 636MB RAM while idle, compared to Outlook Classic sitting around 117MB to 148MB. CPU usage is also worse: around 4% idle on new Outlook versus under 1% on Classic in my testing. Microsoft shut down Mail and Calendar, keeps pushing enterprises toward this web wrapper, and still wants people to believe it is the future of email on Windows. SHAME!

Windows Latest

101,430 views • 1 month ago

1/5 𝗧𝗵𝗲 𝗪𝗼𝗿𝗹𝗱'𝘀 𝗙𝗶𝗿𝘀𝘁 𝗢𝗻-𝗰𝗵𝗮𝗶𝗻 𝗧𝗼𝗸𝗲𝗻𝗶𝘇𝗲𝗱 𝗣𝗼𝗿𝘁𝗳𝗼𝗹𝗶𝗼𝘀 (𝗢𝗧𝗣𝘀) 𝗯𝘆 𝗜𝗫 𝗦𝘄𝗮𝗽! 🚀 #OTPs provide access to a diversified portfolio of assets managed by an investment professional. These assets range from digital assets like crypto and tokens to real estate, private equity, and more. Essentially, its a tokenized fund that can be quickly set up at minimal cost. Investors can begin with just 1 USDT, and all transactions are conducted on-chain using stablecoins. After funding closes, investors receive real-world asset tokens (fund shares) directly in their wallet and can immediately trade on the secondary market (IXS DEX). It’s a cost-effective, fast, and flexible alternative to traditional funds, with no long lock-ups.

1/5 𝗧𝗵𝗲 𝗪𝗼𝗿𝗹𝗱'𝘀 𝗙𝗶𝗿𝘀𝘁 𝗢𝗻-𝗰𝗵𝗮𝗶𝗻 𝗧𝗼𝗸𝗲𝗻𝗶𝘇𝗲𝗱 𝗣𝗼𝗿𝘁𝗳𝗼𝗹𝗶𝗼𝘀 (𝗢𝗧𝗣𝘀) 𝗯𝘆 𝗜𝗫 𝗦𝘄𝗮𝗽! 🚀 #OTPs provide access to a diversified portfolio of assets managed by an investment professional. These assets range from digital assets like crypto and tokens to real estate, private equity, and more. Essentially, its a tokenized fund that can be quickly set up at minimal cost. Investors can begin with just 1 USDT, and all transactions are conducted on-chain using stablecoins. After funding closes, investors receive real-world asset tokens (fund shares) directly in their wallet and can immediately trade on the secondary market (IXS DEX). It’s a cost-effective, fast, and flexible alternative to traditional funds, with no long lock-ups.

IXS

11,094 views • 1 year ago

1/3 🤖 Meet AgentOS: A Token-efficient, Microkernel AI agent with on-device model routing across CLI, Web UI, and chat. A local router reads every message on your device and sends it to the cheapest model that can still do the job well. You stop overpaying for AI. ⚡ What makes it stand out: ■ Smart on-device model routing across 20+ providers ( bankrbot LLM Gateway, OpenRouter, OpenAI, Anthropic, Ollama, and more). ■ Persistent local memory that survives restarts. ■ A layered security sandbox (Standard, Strict, Locked). ■ 37 built-in skills plus MCP, loaded only when a task needs them, and more ■ One unified gateway for CLI, Web UI, Slack, Telegram, Discord, and more Remember this: AgentOS has been integrated with the Bankr LLM Gateway since day one. That means any Bankr user with Bankr API key can start using AgentOS in minutes. Let the router cook.

1/3 🤖 Meet AgentOS: A Token-efficient, Microkernel AI agent with on-device model routing across CLI, Web UI, and chat. A local router reads every message on your device and sends it to the cheapest model that can still do the job well. You stop overpaying for AI. ⚡ What makes it stand out: ■ Smart on-device model routing across 20+ providers ( bankrbot LLM Gateway, OpenRouter, OpenAI, Anthropic, Ollama, and more). ■ Persistent local memory that survives restarts. ■ A layered security sandbox (Standard, Strict, Locked). ■ 37 built-in skills plus MCP, loaded only when a task needs them, and more ■ One unified gateway for CLI, Web UI, Slack, Telegram, Discord, and more Remember this: AgentOS has been integrated with the Bankr LLM Gateway since day one. That means any Bankr user with Bankr API key can start using AgentOS in minutes. Let the router cook.

AgentOS

192,535 views • 26 days ago

Brilliant open-source tool from Anthropic Visualize internal LLM pathways through open attribution graphs. → Attribution graphs reveal which neurons and features drive each output token, turning the LLM’s black box into a clear map. They open-sourced a library that generates attribution graphs to trace how LLMs arrive at outputs. These graphs reveal internal activation flows and feature influences in popular open-weights models. 🧠 Attribution Graphs Graphs link model activations to output tokens. Nodes represent neurons or features. Edges show influence strength. Researchers inspect paths to see which components drive each decision. 🛠️ Toolchain and Frontend Circuit-tracer library works with models like Gemma-2-2b and Llama-3.2-1b. Demo notebooks illustrate multi-step reasoning examples. Neuronpedia hosts an interactive UI for graph exploration, annotation, and sharing. 🔍 Hypothesis Testing Users tweak feature values in graphs, regenerate outputs, and observe changes. This lets researchers validate whether specific circuits control behaviors like language translation or reasoning chains.

Brilliant open-source tool from Anthropic Visualize internal LLM pathways through open attribution graphs. → Attribution graphs reveal which neurons and features drive each output token, turning the LLM’s black box into a clear map. They open-sourced a library that generates attribution graphs to trace how LLMs arrive at outputs. These graphs reveal internal activation flows and feature influences in popular open-weights models. 🧠 Attribution Graphs Graphs link model activations to output tokens. Nodes represent neurons or features. Edges show influence strength. Researchers inspect paths to see which components drive each decision. 🛠️ Toolchain and Frontend Circuit-tracer library works with models like Gemma-2-2b and Llama-3.2-1b. Demo notebooks illustrate multi-step reasoning examples. Neuronpedia hosts an interactive UI for graph exploration, annotation, and sharing. 🔍 Hypothesis Testing Users tweak feature values in graphs, regenerate outputs, and observe changes. This lets researchers validate whether specific circuits control behaviors like language translation or reasoning chains.

Rohan Paul

27,197 views • 1 year ago

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.

AlphaSignal

92,002 views • 4 months ago

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.

A peanut-sized Chinese model just dethroned Gemini at reading documents. GLM-OCR is a 0.9B parameter vision-language model. It scores 94.62 on OmniDocBench V1.5, ranking #1 overall. For context, it outperforms models 100x its size. 100% open-source. It works in two stages. 1. A layout engine detects every region in a document. 2. Each region gets read in parallel. The model predicts multiple tokens per step instead of one. That's what makes it so fast at small size. It handles things most OCR tools struggle with: > Complex tables and nested layouts > Handwritten text and stamps > Math formulas and code blocks > Mixed image-and-text documents You can run it locally through Ollama. It fits on edge devices with limited compute. Every expensive OCR API just got a free competitor.

Jafar Najafov

13,630 views • 3 months ago

1-bit Kimi K3 performs at Opus 5 level on 3D physics! We ran our Atomic Chat quant of Kimi K3 locally on 4x B200 against three cloud models and gave them all the same task, to build a giant anvil drop test as a single HTML file with real physics Outputs: K3 1bit (local): 15.8K tokens, $0 API cost Kimi K3 (API): 15.3K tokens, $0.30 API cost Opus 5: 22.8K tokens, $0.77 API cost GPT 5.6: 14.5K tokens, $0.72 API cost All four got the physics right. But only Kimi made a working winch. The drum turns and the chain drags the flat car off the pad. Opus 5 drew the most detail, road markings and sparks on the hit. And you can run a model at this level on your own box now. That still feels insane to us

1-bit Kimi K3 performs at Opus 5 level on 3D physics! We ran our Atomic Chat quant of Kimi K3 locally on 4x B200 against three cloud models and gave them all the same task, to build a giant anvil drop test as a single HTML file with real physics Outputs: K3 1bit (local): 15.8K tokens, $0 API cost Kimi K3 (API): 15.3K tokens, $0.30 API cost Opus 5: 22.8K tokens, $0.77 API cost GPT 5.6: 14.5K tokens, $0.72 API cost All four got the physics right. But only Kimi made a working winch. The drum turns and the chain drags the flat car off the pad. Opus 5 drew the most detail, road markings and sparks on the hit. And you can run a model at this level on your own box now. That still feels insane to us

atomic.chat

52,841 views • 9 days ago

🚨 Do you understand what Claude just quietly dropped while everyone was distracted? 1 million tokens. Let me explain what that actually means because the number alone doesn't hit right. > A senior engineer joins a company and spends 3 to 6 months just reading code.. Understanding how things connect. Learning where the bugs hide. Why that one file nobody touches exists. It takes months because a codebase is massive and human memory is small. > Claude just loaded the entire thing in one prompt. 30 seconds. Every file, Every function, Every line. All of it. Sitting in memory like it's been working there for years. And it scored highest among every single frontier model. Not GPT.. Not Gemini, Nobody. > Yesterday Amazon's AI nuked production because it couldn't see the full picture - it made a decision with partial context and deleted everything. Today an AI can hold 1 million tokens of context at once. That's the fix. That's the "before and after" moment for AI coding. > 600 images in one request. Entire PDFs. Full repos. And they dropped it on a Friday on all plans like it was a patch note. The scariest AI updates aren't the ones with press conferences. They're the ones that drop in a tweet at 6pm and change everything by Monday morning.

🚨 Do you understand what Claude just quietly dropped while everyone was distracted? 1 million tokens. Let me explain what that actually means because the number alone doesn't hit right. > A senior engineer joins a company and spends 3 to 6 months just reading code.. Understanding how things connect. Learning where the bugs hide. Why that one file nobody touches exists. It takes months because a codebase is massive and human memory is small. > Claude just loaded the entire thing in one prompt. 30 seconds. Every file, Every function, Every line. All of it. Sitting in memory like it's been working there for years. And it scored highest among every single frontier model. Not GPT.. Not Gemini, Nobody. > Yesterday Amazon's AI nuked production because it couldn't see the full picture - it made a decision with partial context and deleted everything. Today an AI can hold 1 million tokens of context at once. That's the fix. That's the "before and after" moment for AI coding. > 600 images in one request. Entire PDFs. Full repos. And they dropped it on a Friday on all plans like it was a patch note. The scariest AI updates aren't the ones with press conferences. They're the ones that drop in a tweet at 6pm and change everything by Monday morning.

Tuki

206,260 views • 4 months ago

Right now, you may not have access to models like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.

Right now, you may not have access to models like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.

Alok

65,387 views • 1 month ago

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,840,585 views • 3 months ago

GPT-5.6 Sol is unbelievably good at creating and editing videos. It can do motion design, product demos, and animations like this one I made by simply giving it a screen recording. GPT 5.6 has the best design taste and significantly outperforms Fable, which relies heavily on repetitive design patterns. To help you experiment with video editing on it, we just launched a collection of 100 ready-to-use skills that show what’s possible and help you get started with video editing using GPT-5.6. These skills can create anything from motion graphics launch videos for your product to a 3B1B-style science explainer video. You can also use them to edit existing videos: add captions, generate motion graphics, create voiceovers, redesign visual styles, translate into new languages, and much more. If you want access to the full library, comment “VIDEO SKILLS” and I’ll share it with you. (You'll have to follow me so I can DM you.)

GPT-5.6 Sol is unbelievably good at creating and editing videos. It can do motion design, product demos, and animations like this one I made by simply giving it a screen recording. GPT 5.6 has the best design taste and significantly outperforms Fable, which relies heavily on repetitive design patterns. To help you experiment with video editing on it, we just launched a collection of 100 ready-to-use skills that show what’s possible and help you get started with video editing using GPT-5.6. These skills can create anything from motion graphics launch videos for your product to a 3B1B-style science explainer video. You can also use them to edit existing videos: add captions, generate motion graphics, create voiceovers, redesign visual styles, translate into new languages, and much more. If you want access to the full library, comment “VIDEO SKILLS” and I’ll share it with you. (You'll have to follow me so I can DM you.)

Akash Anand

512,538 views • 1 month ago

Create a short film like this in just 1 minute with GPT Image 2.0 + Seedance 2.0. GPT Image 2.0 can naturally combine multiple photos into one single image, while Seedance 2.0 can use that image as a reference to automatically separate the scenes, generate a coherent video sequence, and add suitable background music. This workflow greatly improves the overall creative efficiency. When using this method, simply provide the merged image as a reference for Seedance 2.0 and briefly describe each scene with a simple prompt. This can significantly increase the success rate of the final video. All of the above was created on GPT Image Prompt: Seedance Prompt:

Create a short film like this in just 1 minute with GPT Image 2.0 + Seedance 2.0. GPT Image 2.0 can naturally combine multiple photos into one single image, while Seedance 2.0 can use that image as a reference to automatically separate the scenes, generate a coherent video sequence, and add suitable background music. This workflow greatly improves the overall creative efficiency. When using this method, simply provide the merged image as a reference for Seedance 2.0 and briefly describe each scene with a simple prompt. This can significantly increase the success rate of the final video. All of the above was created on GPT Image Prompt: Seedance Prompt:

Midjourney Sref and prompt Library

40,572 views • 3 months ago

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Avi Chawla

89,234 views • 1 month ago

🚨 Insider wallet alert: The same wallet that nailed the exact timing of the US–Iran strike with a $500K bet just dropped $700K on a ground invasion of Iran. I built a step‑by‑step guide to track and copy‑trade insider wallets like this on Polymarket. Free for 24 hours: - Comment: '' Market '' - Like + Retweet - Follow Theo (so I can DM you) All you need: Claude + laptop + 1 hour/day. While you’re still “analysing charts,” this wallet is using AI and betting millions before headlines hit CNN. They don’t guess. They don’t hesitate. They move first—and win. You can keep watching from the side-lines… or learn how to move with them. Don't forget to bookmark and follow me Theo

🚨 Insider wallet alert: The same wallet that nailed the exact timing of the US–Iran strike with a $500K bet just dropped $700K on a ground invasion of Iran. I built a step‑by‑step guide to track and copy‑trade insider wallets like this on Polymarket. Free for 24 hours: - Comment: '' Market '' - Like + Retweet - Follow Theo (so I can DM you) All you need: Claude + laptop + 1 hour/day. While you’re still “analysing charts,” this wallet is using AI and betting millions before headlines hit CNN. They don’t guess. They don’t hesitate. They move first—and win. You can keep watching from the side-lines… or learn how to move with them. Don't forget to bookmark and follow me Theo

Theo

27,827 views • 4 months ago

A 17-year-old student spent $4,200 on 7 Mac minis. Small silver boxes. Stacked on a desk. Connected in one room. From the outside, it looked like a stupid purchase. But inside, it wasn't just 7 computers. It was Skills. Hooks. Memory. Worktrees. One machine handled repeatable tasks. One ran checks automatically. One kept context between sessions. Others ran parallel jobs without touching each other's work. While most people were still typing the same instructions again and again, his setup was already moving. A lot of people pay $200 a month for Claude and still use maybe 20% of it. He built a system around it. Skills turned repeated work into reusable workflows. Hooks made actions fire automatically. Memory stopped every session from starting at zero. Worktrees let multiple tasks run at the same time without collisions. That changed everything. Setup time: under 1 hour once. Time returned: 3 to 5 hours every day. He spent $4,200 once. He made $16,000 in the first week. Not because he found a secret tool. Not because he wrote magical prompts. Because he stopped using it like a chatbot and started using it like infrastructure. 7 Mac minis. 1 student. $4,200 in. $16,000 out. And most people would still call it just a stack of computers.

A 17-year-old student spent $4,200 on 7 Mac minis. Small silver boxes. Stacked on a desk. Connected in one room. From the outside, it looked like a stupid purchase. But inside, it wasn't just 7 computers. It was Skills. Hooks. Memory. Worktrees. One machine handled repeatable tasks. One ran checks automatically. One kept context between sessions. Others ran parallel jobs without touching each other's work. While most people were still typing the same instructions again and again, his setup was already moving. A lot of people pay $200 a month for Claude and still use maybe 20% of it. He built a system around it. Skills turned repeated work into reusable workflows. Hooks made actions fire automatically. Memory stopped every session from starting at zero. Worktrees let multiple tasks run at the same time without collisions. That changed everything. Setup time: under 1 hour once. Time returned: 3 to 5 hours every day. He spent $4,200 once. He made $16,000 in the first week. Not because he found a secret tool. Not because he wrote magical prompts. Because he stopped using it like a chatbot and started using it like infrastructure. 7 Mac minis. 1 student. $4,200 in. $16,000 out. And most people would still call it just a stack of computers.

Gipp 🦅

21,280 views • 3 months ago

This $50,000 looking website was built with Claude, the product? A wooden cup coaster. Apple-keynote dark theme with cinematic lighting: ORYZO PRO/PRO MAX configurator, “37.9% more circular” and “Powered by AI” on the homepage, hand holding real wood. Design agencies charge $50,000 minimum for sites this clean, take 8 weeks to deliver and require 6 rounds of revisions before approval. This guy typed prompts and got a launch page that looks better than most real tech companies running real funding rounds. The product is a wooden disc that costs $2 to make, the website looks like it belongs to a $10 billion startup. Claude doesn't care what you're selling, only how good the page can look. The barrier between an idea and an Apple-level launch just dropped to a few prompts. And some people still say, "To run a business, you need money"

This $50,000 looking website was built with Claude, the product? A wooden cup coaster. Apple-keynote dark theme with cinematic lighting: ORYZO PRO/PRO MAX configurator, “37.9% more circular” and “Powered by AI” on the homepage, hand holding real wood. Design agencies charge $50,000 minimum for sites this clean, take 8 weeks to deliver and require 6 rounds of revisions before approval. This guy typed prompts and got a launch page that looks better than most real tech companies running real funding rounds. The product is a wooden disc that costs $2 to make, the website looks like it belongs to a $10 billion startup. Claude doesn't care what you're selling, only how good the page can look. The barrier between an idea and an Apple-level launch just dropped to a few prompts. And some people still say, "To run a business, you need money"

Defileo🔮

313,744 views • 3 months ago

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. I wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. The article is quoted below.

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. I wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. The article is quoted below.

Akshay 🚀

152,917 views • 1 day ago