正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

We got 755 tokens per second! 🔥 That's OpenMed privacy-filter v2 (nemotron, MLX 8-bit) reading a 13,000-token clinical file and redacting every identifier as it streams by. 1,152 caught across 22 PII categories, on a Mac. Nothing left the machine.

Maziyar PANAHI

18,723 subscribers

147,213 次观看 • 21 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

OpenHands crushed Codex by 2.3× on token efficiency! We gave both agents the same task on the same model (Qwen3.5 35B): build an 8-bit Space Invaders in 3 iterations (build, fix, polish). Output: • OpenHands: 219K tokens, • Codex: 513K tokens, OpenHands beats Codex in local running. The difference is in how they handle context. OpenHands reuses the unchanged data across every pass and only pays for new tokens, while Codex re-sends and re-counts it every iteration. OpenHands finished a bit slower, but on local runs, tokens spent are the real cost.

OpenHands crushed Codex by 2.3× on token efficiency! We gave both agents the same task on the same model (Qwen3.5 35B): build an 8-bit Space Invaders in 3 iterations (build, fix, polish). Output: • OpenHands: 219K tokens, • Codex: 513K tokens, OpenHands beats Codex in local running. The difference is in how they handle context. OpenHands reuses the unchanged data across every pass and only pays for new tokens, while Codex re-sends and re-counts it every iteration. OpenHands finished a bit slower, but on local runs, tokens spent are the real cost.

atomic.chat

28,088 次观看 • 8 天前

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less

Noisy

185,148 次观看 • 1 个月前

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓

slash1s

101,133 次观看 • 1 个月前

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables

Noisy

48,354 次观看 • 1 个月前

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

Noisy

79,665 次观看 • 1 个月前

Nvidia CEO Jensen Huang on why engineers will soon be paid in tokens, not just salary: Jensen lays out a future where compute access becomes part of an engineer's compensation package. "I could totally imagine in the future every single engineer in our company will need an annual token budget," he says. He explains how the math would work: "They're going to make a few hundred thousand a year their base pay. I'm going to give them probably half of that on top of it as tokens so that they could be amplified 10x. Of course, we would." According to Huang, this is already changing how companies compete for talent: "It is now one of the recruiting tools in Silicon Valley. How many tokens comes along with my job?" His reasoning is simple: tokens make engineers more productive. As he puts it, "every engineer that has access to tokens will be more productive and those tokens as you know will be produced by AI factories that all of you and us we partner to build." Huang then zooms out to describe how this reshapes the nature of companies themselves: "Every single enterprise company in today sit on top of file systems and data centers. Every single software company of the future will be agentic and they will be token manufacturers. They be token users for their engineers and they'll be token manufacturers for all of their customers."

Nvidia CEO Jensen Huang on why engineers will soon be paid in tokens, not just salary: Jensen lays out a future where compute access becomes part of an engineer's compensation package. "I could totally imagine in the future every single engineer in our company will need an annual token budget," he says. He explains how the math would work: "They're going to make a few hundred thousand a year their base pay. I'm going to give them probably half of that on top of it as tokens so that they could be amplified 10x. Of course, we would." According to Huang, this is already changing how companies compete for talent: "It is now one of the recruiting tools in Silicon Valley. How many tokens comes along with my job?" His reasoning is simple: tokens make engineers more productive. As he puts it, "every engineer that has access to tokens will be more productive and those tokens as you know will be produced by AI factories that all of you and us we partner to build." Huang then zooms out to describe how this reshapes the nature of companies themselves: "Every single enterprise company in today sit on top of file systems and data centers. Every single software company of the future will be agentic and they will be token manufacturers. They be token users for their engineers and they'll be token manufacturers for all of their customers."

Big Brain AI

90,171 次观看 • 2 个月前

TOKENFI TOKEN LAUNCHER IS LIVE ON MAINNET IT’S OFFICIAL! TokenFi Token Launcher is live on the mainnet of five top EVM blockchains: Ethereum, BNB Chain, Base, Arbitrum, and opBNB. With TokenFi Token Launcher, ANYONE can create a fungible crypto token (ERC-20), NFT (ERC-721), or Multi-Token (ERC-1155) in just a few clicks, regardless of their crypto experience. The TokenFi platform and framework has been thoroughly audited and is being monitored in real-time by industry-leading blockchain security auditor Certik. This means that the tokens you create with TokenFi are safe and audited. TokenFi Token Launcher is powered by $TOKEN as its main utility token, and every successful transaction on the Ethereum and BNB blockchains buys and burns $TOKEN, enhancing its utility and making it perpetually deflationary as adoption of the platform grows. TokenFi Token Launcher aims to be the dominant token creation platform on EVM blockchains, and to this effect, we will be rolling out the platform on dozens of additional EVM blockchains in the coming weeks and months. You can learn more about TokenFi Token Launcher here:

TOKENFI TOKEN LAUNCHER IS LIVE ON MAINNET IT’S OFFICIAL! TokenFi Token Launcher is live on the mainnet of five top EVM blockchains: Ethereum, BNB Chain, Base, Arbitrum, and opBNB. With TokenFi Token Launcher, ANYONE can create a fungible crypto token (ERC-20), NFT (ERC-721), or Multi-Token (ERC-1155) in just a few clicks, regardless of their crypto experience. The TokenFi platform and framework has been thoroughly audited and is being monitored in real-time by industry-leading blockchain security auditor Certik. This means that the tokens you create with TokenFi are safe and audited. TokenFi Token Launcher is powered by $TOKEN as its main utility token, and every successful transaction on the Ethereum and BNB blockchains buys and burns $TOKEN, enhancing its utility and making it perpetually deflationary as adoption of the platform grows. TokenFi Token Launcher aims to be the dominant token creation platform on EVM blockchains, and to this effect, we will be rolling out the platform on dozens of additional EVM blockchains in the coming weeks and months. You can learn more about TokenFi Token Launcher here:

TokenFi

1,288,892 次观看 • 2 年前

WHOS READY TO RUN IT BACK TURBO?! Introducing FUTURE DAO! So we got DAOs ✅ We got Prediction markets ✅ We got Ai ✅ We got meme-coins✅ We got a pre sale✅ Prediction markets have been a bit boring to me TBH, I WANT TO HIT 100Xs not like do 10hrs of research on Greenland politics or something. So I created this DAO to give people some leverage, and make it fun. Basically how it works, is we do a pre sale, people get tokens, the rest of the token supply goes into the DAO. Like a classic DAOs fun but you can redeem the AUM at any time,(but you incur a penalty) . You have the AUM, and our proprietary Ai does the trading for you. Just hold the token and watch it go up(hopefully), the Ai trades with guidance from Token holder votes. The token holders also get to vote/guide the Ai into outcomes they think will benefit the DAO/token. AUM Goes up. Token Goes up. Everyone happy. Or AUM flat/Down. Token Goes down. People Redeem and get % of their money back. better than zeroing out on meme coins. The token will all be launched via a semi-gated pre sale, And using my own method of launching to showcase how its done and to ensure there is a good group of "seed DAO" members to help the Ai. it will run 8 weeks, and people can decide to shut it down and get $$ back. or keep it going forever. end of the day this is a meme coin guys, but I think its a cool experiment.

WHOS READY TO RUN IT BACK TURBO?! Introducing FUTURE DAO! So we got DAOs ✅ We got Prediction markets ✅ We got Ai ✅ We got meme-coins✅ We got a pre sale✅ Prediction markets have been a bit boring to me TBH, I WANT TO HIT 100Xs not like do 10hrs of research on Greenland politics or something. So I created this DAO to give people some leverage, and make it fun. Basically how it works, is we do a pre sale, people get tokens, the rest of the token supply goes into the DAO. Like a classic DAOs fun but you can redeem the AUM at any time,(but you incur a penalty) . You have the AUM, and our proprietary Ai does the trading for you. Just hold the token and watch it go up(hopefully), the Ai trades with guidance from Token holder votes. The token holders also get to vote/guide the Ai into outcomes they think will benefit the DAO/token. AUM Goes up. Token Goes up. Everyone happy. Or AUM flat/Down. Token Goes down. People Redeem and get % of their money back. better than zeroing out on meme coins. The token will all be launched via a semi-gated pre sale, And using my own method of launching to showcase how its done and to ensure there is a good group of "seed DAO" members to help the Ai. it will run 8 weeks, and people can decide to shut it down and get $$ back. or keep it going forever. end of the day this is a meme coin guys, but I think its a cool experiment.

Skely

47,630 次观看 • 6 个月前

🔥 NVIDIA GTC 2026: The Vera Rubin platform just dropped, introducing 10x inference throughput per watt, at 1/10th the cost per token vs. Blackwell. Jensen Huang called it: every company will now run a "token factory." Here's the thing: Aethir has been running distributed token factories since day one, with nearly 440,000+ GPU containers across 94 countries, delivering 1.5B+ compute hours to enterprises, powered by ATH. When the world's most powerful AI chips land, you need a cloud that can actually deploy them at scale, without the centralized cloud limitations and bottlenecks. That's Aethir’s advantage over centralized compute providers. ⚡

🔥 NVIDIA GTC 2026: The Vera Rubin platform just dropped, introducing 10x inference throughput per watt, at 1/10th the cost per token vs. Blackwell. Jensen Huang called it: every company will now run a "token factory." Here's the thing: Aethir has been running distributed token factories since day one, with nearly 440,000+ GPU containers across 94 countries, delivering 1.5B+ compute hours to enterprises, powered by ATH. When the world's most powerful AI chips land, you need a cloud that can actually deploy them at scale, without the centralized cloud limitations and bottlenecks. That's Aethir’s advantage over centralized compute providers. ⚡

Aethir Ecosystem

14,360 次观看 • 4 个月前

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

268,010 次观看 • 29 天前

What Happens When AI Tokens Cost More Than Your Employees? @jason: “We, with our agents, hit $300/day per agent using the Claude API, like instantly. And that was doing, maybe, 10 or 20%. That's $100k/year per agent.” Chamath Palihapitiya: “We're getting to a place where we have to basically now say, ‘What is the token budget that we're willing to give our best devs?’” “And then if you aggregate it across all people, you can clearly see a trend where you're like, ‘Well, hold on a second, now they need to be at least 2x as productive as another employee.’” “That is actively happening inside my business, because otherwise I'll run out of money.” Jason: “Yeah. This is a very interesting trend that you're not going to hear anybody else talk about, but when do tokens outpace the salary of the employee?” “Because you're about to hit it. I'm about to hit it.”

What Happens When AI Tokens Cost More Than Your Employees? @jason: “We, with our agents, hit $300/day per agent using the Claude API, like instantly. And that was doing, maybe, 10 or 20%. That's $100k/year per agent.” Chamath Palihapitiya: “We're getting to a place where we have to basically now say, ‘What is the token budget that we're willing to give our best devs?’” “And then if you aggregate it across all people, you can clearly see a trend where you're like, ‘Well, hold on a second, now they need to be at least 2x as productive as another employee.’” “That is actively happening inside my business, because otherwise I'll run out of money.” Jason: “Yeah. This is a very interesting trend that you're not going to hear anybody else talk about, but when do tokens outpace the salary of the employee?” “Because you're about to hit it. I'm about to hit it.”

The All-In Podcast

2,714,868 次观看 • 5 个月前

$A $351 MINI PC IS RUNNING 26-BILLION-PARAMETER AI MODELS AT 20 TOKENS/SEC AND HERMES AGENT ON TOP OF IT This is the Minisforum UM790 Pro. AMD Ryzen 9 7940HS, Radeon 780M iGPU, 48GB DDR5. The BIOS reports the GPU has 4GB of VRAM Here's the part people get wrong. The 780M has no dedicated VRAM at all it borrows from system RAM. Vulkan ignores the BIOS number and reads the full 48GB pool directly That's the whole trick. 21+ GB allocated to model weights on a "4GB" GPU. Because the models are MoE, only 4 billion of the parameters activate per token a fraction of the reads a dense model needs. That's why 20 tok/s works here Gemma 4 26B MoE holds 19.5 tok/s with 196K context. Qwen3.5-35B-A3B holds 20.8. Nemotron Cascade 2 clears 24.8. A dense 31B, by contrast, drops to 4 tok/s it reads the entire model every token, no way around it On top: Hermes Agent, full agentic workflows terminal, file ops, web, 40+ tools against local models only. No API keys. The wall between you and a usable local agent used to be a GPU you couldn't afford. Now it's a BIOS setting most people never check Bookmark this & Try it yourself ↓$

A $351 MINI PC IS RUNNING 26-BILLION-PARAMETER AI MODELS AT 20 TOKENS/SEC AND HERMES AGENT ON TOP OF IT This is the Minisforum UM790 Pro. AMD Ryzen 9 7940HS, Radeon 780M iGPU, 48GB DDR5. The BIOS reports the GPU has 4GB of VRAM Here's the part people get wrong. The 780M has no dedicated VRAM at all it borrows from system RAM. Vulkan ignores the BIOS number and reads the full 48GB pool directly That's the whole trick. 21+ GB allocated to model weights on a "4GB" GPU. Because the models are MoE, only 4 billion of the parameters activate per token a fraction of the reads a dense model needs. That's why 20 tok/s works here Gemma 4 26B MoE holds 19.5 tok/s with 196K context. Qwen3.5-35B-A3B holds 20.8. Nemotron Cascade 2 clears 24.8. A dense 31B, by contrast, drops to 4 tok/s it reads the entire model every token, no way around it On top: Hermes Agent, full agentic workflows terminal, file ops, web, 40+ tools against local models only. No API keys. The wall between you and a usable local agent used to be a GPU you couldn't afford. Now it's a BIOS setting most people never check Bookmark this & Try it yourself ↓

slash1s

35,742 次观看 • 1 个月前

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks

Noisy

21,553 次观看 • 1 个月前

november 15th 1 year ago today, the chillguy token graduated from pumpfun as a meme, it changed the meaning of going viral. it was used by almost every celebrity, athlete, and brand across every demographic and culture. a simple character, yet so relatable and inspiring. as a memecoin, it was the first to succeed in mass onboarding new users. 148,000+ holders in the span of two weeks, not even including the countless holders on exchanges. the whole world was watching us in awe. the market today looks different from how it did one year ago, but the chill guy community and the relatability of the meme haven’t changed one bit. if anything, our mission of spreading positivity and ‘chillness’ across the world & to serve as a safe and relatable onboarding tool to our fantastic industry are even more important now. we will simply keep on chilling, together.

just a chill guy | fan page

15,785 次观看 • 8 个月前

the tiebreaker is done. qwen 3.5 27B dense. single RTX 3090. one prompt. zero steering. zero human edits. 1,827 lines across 10 files. 13 minutes. full thinking mode. runs on first load. hermes 4.3 got the same prompt with 2x 3090s and 5x the context it needed. wrote 1,249 lines, left empty files, needed 3 interventions, game was broken on load. same architecture class. same quant. hermes got double the hardware. completely different result. dense wasn't the problem. hermes was. but here's what got me. this model thinks at 27 tok/s. every single token carries 27 billion parameters of reasoning. MoE hit 112 tok/s but only 3B active per token. the dense model is slower and it doesn't matter. watch 13 minutes of autonomous coding on a consumer GPU with zero intervention and tell me speed is what matters. a year ago this wasn't possible. now it runs on hardware you can buy used for $900. no API. no subscription. no cloud. just a 3090 doing what data centers did 18 months ago. full unedited session in the video. every token, every file, every thinking chain. 16 minutes. hit play.

the tiebreaker is done. qwen 3.5 27B dense. single RTX 3090. one prompt. zero steering. zero human edits. 1,827 lines across 10 files. 13 minutes. full thinking mode. runs on first load. hermes 4.3 got the same prompt with 2x 3090s and 5x the context it needed. wrote 1,249 lines, left empty files, needed 3 interventions, game was broken on load. same architecture class. same quant. hermes got double the hardware. completely different result. dense wasn't the problem. hermes was. but here's what got me. this model thinks at 27 tok/s. every single token carries 27 billion parameters of reasoning. MoE hit 112 tok/s but only 3B active per token. the dense model is slower and it doesn't matter. watch 13 minutes of autonomous coding on a consumer GPU with zero intervention and tell me speed is what matters. a year ago this wasn't possible. now it runs on hardware you can buy used for $900. no API. no subscription. no cloud. just a 3090 doing what data centers did 18 months ago. full unedited session in the video. every token, every file, every thinking chain. 16 minutes. hit play.

Sudo su

91,135 次观看 • 4 个月前

$THIS GUY AUDITED 926 CLAUDE CODE SESSIONS AND FOUND MOST OF THE TOKEN WASTE WAS ON HIS SIDE everyone is blaming anthropic for the limits, so he decided to actually look at the data 858 sessions, 18,903 turns, and $1,619 estimated spend across 33 days here's what he found: 1\ one default setting was burning 14,000 tokens per turn Claude Code loads the full JSON schema for every tool into context at session start. whether you use them or not. 20,000 tokens of tool definitions sitting there on every single turn. the fix: one line in your settings.json "ENABLE_TOOL_SEARCH": "true" context dropped from 45K to 20K instantly. across 858 sessions that one setting was wasting an estimated 264 million tokens 2\ cache expiry is the single biggest waste 54% of his turns came after a 5+ minute idle gap. every one of those turns re-processed the entire conversation at full price which caused a 10x cost jump you go grab coffee. come back 5 minutes later. type your next message. everything rebuilds from scratch. the context didn't change. you didn't change. the cache just expired. 12.3 million tokens wasted on idle gaps alone 3\ 42 skills loaded. 19 of them used twice or less across 858 sessions. every one of those skill schemas sat in context on every turn eating tokens for nothing. 4\ 1,122 redundant file reads where the same file was read 3+ times one session read the same file 33 times. he ALSO built a full token auditor dashboard that shows you exactly where your waste is coming from 19 charts, opens in your browser, free AND open source$

THIS GUY AUDITED 926 CLAUDE CODE SESSIONS AND FOUND MOST OF THE TOKEN WASTE WAS ON HIS SIDE everyone is blaming anthropic for the limits, so he decided to actually look at the data 858 sessions, 18,903 turns, and $1,619 estimated spend across 33 days here's what he found: 1\ one default setting was burning 14,000 tokens per turn Claude Code loads the full JSON schema for every tool into context at session start. whether you use them or not. 20,000 tokens of tool definitions sitting there on every single turn. the fix: one line in your settings.json "ENABLE_TOOL_SEARCH": "true" context dropped from 45K to 20K instantly. across 858 sessions that one setting was wasting an estimated 264 million tokens 2\ cache expiry is the single biggest waste 54% of his turns came after a 5+ minute idle gap. every one of those turns re-processed the entire conversation at full price which caused a 10x cost jump you go grab coffee. come back 5 minutes later. type your next message. everything rebuilds from scratch. the context didn't change. you didn't change. the cache just expired. 12.3 million tokens wasted on idle gaps alone 3\ 42 skills loaded. 19 of them used twice or less across 858 sessions. every one of those skill schemas sat in context on every turn eating tokens for nothing. 4\ 1,122 redundant file reads where the same file was read 3+ times one session read the same file 33 times. he ALSO built a full token auditor dashboard that shows you exactly where your waste is coming from 19 charts, opens in your browser, free AND open source

Om Patel

298,344 次观看 • 3 个月前

most agent memory is one file that grows forever and gets re-read every single turn. it overflows the context, overwrites old facts, loses the thread. that's why your agent feels sharp on day one and lost by week three. Sibyl Memory replaces the pile with a structure: → leaner context, lower token cost. a bloated agent file and flat memory get re-read in full every message. Sibyl Memory keeps a light hot layer and retrieves the rest on demand, so each turn carries only the tokens that matter. → relations are first-class. people link to projects, projects to deals, deals to the decisions behind them. ask about one partner and the agent surfaces the whole connected web around them. that relational context is what lets it help run a company. → one source of truth per fact, person, and project. no duplicates, no silent overwrites. works with hermes, claude code, and codex. beta open.

most agent memory is one file that grows forever and gets re-read every single turn. it overflows the context, overwrites old facts, loses the thread. that's why your agent feels sharp on day one and lost by week three. Sibyl Memory replaces the pile with a structure: → leaner context, lower token cost. a bloated agent file and flat memory get re-read in full every message. Sibyl Memory keeps a light hot layer and retrieves the rest on demand, so each turn carries only the tokens that matter. → relations are first-class. people link to projects, projects to deals, deals to the decisions behind them. ask about one partner and the agent surfaces the whole connected web around them. that relational context is what lets it help run a company. → one source of truth per fact, person, and project. no duplicates, no silent overwrites. works with hermes, claude code, and codex. beta open.

SIBYL

81,361 次观看 • 1 个月前

The Nottingham maternity scandal should shock and shame the NHS. It must be treated as an emergency by every hospital boss across the country. Babies dying unnecessarily. Mothers being treated appallingly. Casual disregard for families. Even babies being referred to as clinical waste by the mortuary service. It’s harrowing and heartbreaking. Staff who saw the failings being ignored. Families fighting the system, and not being believed. I’ve met some of the 2,500 families who came forward, a number of whom are from my constituency. I have nothing but admiration for their willingness to share their deeply personal stories. There’s a case for a national statutory inquiry now, and that should be considered. But my priority is not more reports. We’ve had so many now and the list is growing. There have been scandals in Shrewsbury, Morecambe, Sussex, Leeds… we need action. It is astonishing that the NHS is spending almost as much on clinical negligence claims for maternity services as it does on maternity services themselves. It gives you a sense of the scale of the failure. Every mother, every family, deserves to have confidence in the NHS when giving birth. It’s as simple as that.

The Nottingham maternity scandal should shock and shame the NHS. It must be treated as an emergency by every hospital boss across the country. Babies dying unnecessarily. Mothers being treated appallingly. Casual disregard for families. Even babies being referred to as clinical waste by the mortuary service. It’s harrowing and heartbreaking. Staff who saw the failings being ignored. Families fighting the system, and not being believed. I’ve met some of the 2,500 families who came forward, a number of whom are from my constituency. I have nothing but admiration for their willingness to share their deeply personal stories. There’s a case for a national statutory inquiry now, and that should be considered. But my priority is not more reports. We’ve had so many now and the list is growing. There have been scandals in Shrewsbury, Morecambe, Sussex, Leeds… we need action. It is astonishing that the NHS is spending almost as much on clinical negligence claims for maternity services as it does on maternity services themselves. It gives you a sense of the scale of the failure. Every mother, every family, deserves to have confidence in the NHS when giving birth. It’s as simple as that.

Robert Jenrick

18,267 次观看 • 1 个月前

We raised $3M on one observation: Agents learn on your dime. And forget. Every run, your agent figures out the same APIs, makes the same calls, hits the same failures. Because nothing saved what worked. We built Rote by Modiqo Inc to fix this. Point it at any API. Rote reads the spec and builds a surface your agent can call: auth, endpoints, schemas. That's the adapter. When your agent finishes a task, Rote saves the exact calls that worked. That's the flow. Config in. Code out. Runs on your machine. Nothing to watch over. Agents stop starting from scratch. Teams stop paying for the same work twice. Install →

We raised $3M on one observation: Agents learn on your dime. And forget. Every run, your agent figures out the same APIs, makes the same calls, hits the same failures. Because nothing saved what worked. We built Rote by Modiqo Inc to fix this. Point it at any API. Rote reads the spec and builds a surface your agent can call: auth, endpoints, schemas. That's the adapter. When your agent finishes a task, Rote saves the exact calls that worked. That's the flow. Config in. Code out. Runs on your machine. Nothing to watch over. Agents stop starting from scratch. Teams stop paying for the same work twice. Install →

chetan conikee

468,720 次观看 • 1 个月前

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 次观看 • 2 个月前