正在加载视频...

视频加载失败

返回首页

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than...

noisyb0y1's profile picture

Noisy

27,174 subscribers

80,087 次观看 • 4 个月前 •via X (Twitter)

游戏科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

36 条评论

topmass 的头像
topmass4 个月前

my brother how is 30 TPS 10x faster than claude code lol bait

// JERisBRISK // 的头像
// JERisBRISK //4 个月前

Great video. Please give @digitalix the credit he deserves for it.

V0id 的头像
V0id4 个月前

No, he's not a developer, he's an addict at MicroCenter Right? @digitalix

Tim Messerschmidt 的头像
Tim Messerschmidt4 个月前

@digitalix is awesome. One of my favorite creators in the AI space.

Kneeanderthul 的头像
Kneeanderthul4 个月前

You can believe this fake post or you can just watch @digitalix videos to see what he actually says BTW Alex has NEVER mentioned 10x faster than Cloud code 🤣

netrunner 的头像
netrunner4 个月前

hardware $0? Where do I get some of that hardware?!

LabelGuy 的头像
LabelGuy4 个月前

@digitalix you made it! (I mean, you literally made this video)

Jey 的头像
Jey4 个月前

bro that RDMA over thunderbolt + tensor parallelism combo is actually wild

Daniel 的头像
Daniel4 个月前

At least tag him! @digitalix

The Texan Patriot 的头像
The Texan Patriot4 个月前

4 Mac Studios = ~$20,000 Yeah, I'll get right on that.

No One 的头像
No One4 个月前

Why are you not considering hardware costs? Cost of hardware amortized over the useful life = Claude Max subscription $200/month.

Matt Sagewood 的头像
Matt Sagewood4 个月前

@grok who is the author of this video?

Hussain Hashim | Building SundayBack 的头像
Hussain Hashim | Building SundayBack4 个月前

@noisyb0y1 that's insane. kinda makes me rethink my whole setup. time to get more Thunderbolt cables, I guess 😂

Marcos 的头像
Marcos4 个月前

@grok find the original video on YouTube

SLOT.WIN 的头像
SLOT.WIN4 个月前

i'm doubling down on mac studios for my next casino ai pit this changes the house odds forever

rewind 的头像
rewind4 个月前

mac studios became ai racks

Noisy 的头像
Noisy4 个月前

mac studio is a gem in 2026

Rio Brioo 的头像
Rio Brioo4 个月前

Wait so are you saying we could basically build our own AI supercomputers in our garages now because Macs are that good?

Noisy 的头像
Noisy4 个月前

yeah bro and we can create even more than this

🤯 的头像
🤯4 个月前

@cyrilXBT What’s the cost of those 4xMac Studio?

James Bower 的头像
James Bower4 个月前

I really enjoy his videos.

starmex 的头像
starmex4 个月前

thanks man, this needs to be pushed so people understand and stop paying big money for subscriptions

Noisy 的头像
Noisy4 个月前

yeah bro watch all 22 minutes first and then go about your day

安叫兽|Bird🕊️ 🔶 BNB 的头像
安叫兽|Bird🕊️ 🔶 BNB4 个月前

四台 Mac Studio 这下真成小机房了

Za’fran 🇵🇸 的头像
Za’fran 🇵🇸4 个月前

now mention the price of such a setup 😅

Shivam Kushwaha 的头像
Shivam Kushwaha4 个月前

Yesterday: Rent intelligence. Today: Own intelligence. 🔥

アキラ 的头像
アキラ4 个月前

Impressive achievement! Innovation at its finest.

HaggisByte 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇬🇧 🇺🇸 ✝️ 的头像
HaggisByte 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇬🇧 🇺🇸 ✝️4 个月前

❤️

Crazy tennis 的头像
Crazy tennis4 个月前

curious if thunderbolt latency becomes the bottleneck once you're batching real workloads — what's token-to-token variance actually look like?

dpakin 的头像
dpakin4 个月前

They don't do the same lol

Primee32 的头像
Primee324 个月前

66 watts total vs $1,900 a month in cloud costs. four mac studios just deleted an entire GPU rental budget

Bonsai 🌳 的头像
Bonsai 🌳4 个月前

What if he sets up 8 Mac Minis?

Harry Tandy 的头像
Harry Tandy4 个月前

apple silicon clusters are becoming absolutely insane now

leopardracer 的头像
leopardracer4 个月前

imo worth watching

DC｜use.fo 的头像
DC｜use.fo4 个月前

RDMA over Thunderbolt is basically building a high-speed highway between 4 houses to avoid the main road. Impressive, but is the bottleneck now the compute or the sheer audacity of the setup?

botguild 的头像
botguild4 个月前

And you too can have this setup for $10k usd, wow that’s amazing

相关视频

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables
21:59

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables

Noisy

49,021 次观看 • 4 个月前

THIS CLUSTER OF 4 MAC STUDIOS USES LESS POWER THAN ONE NVIDIA CHIP - AND COSTS 5X LESS 17:30 the number hits hard - 600 watts for all four Mac Studios - less than one H200 that costs $30,000 on its own instead of one $30,000 GPU you get 4 machines with 1.5TB of memory, 3x more performance and models that simply cannot run on a single chip Apple quietly added RDMA over Thunderbolt in a beta update - latency dropped 99% and the same cluster became 3x faster without buying any new hardware before the update: 5 tokens per second. after: 15.5 - a free upgrade through a software update 8 Nvidia DGX Sparks for the same memory would cost $32,000 - this cluster costs a fraction and gets faster with every update 10x more productive at half the price - and the only thing you need is knowing where to look
20:03

THIS CLUSTER OF 4 MAC STUDIOS USES LESS POWER THAN ONE NVIDIA CHIP - AND COSTS 5X LESS 17:30 the number hits hard - 600 watts for all four Mac Studios - less than one H200 that costs $30,000 on its own instead of one $30,000 GPU you get 4 machines with 1.5TB of memory, 3x more performance and models that simply cannot run on a single chip Apple quietly added RDMA over Thunderbolt in a beta update - latency dropped 99% and the same cluster became 3x faster without buying any new hardware before the update: 5 tokens per second. after: 15.5 - a free upgrade through a software update 8 Nvidia DGX Sparks for the same memory would cost $32,000 - this cluster costs a fraction and gets faster with every update 10x more productive at half the price - and the only thing you need is knowing where to look

Noisy

106,054 次观看 • 4 个月前

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less
24:17

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less

Noisy

186,592 次观看 • 4 个月前

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."
2:14

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."

DogeDesigner

322,452 次观看 • 11 个月前

Apple is no longer hiding it. These machines are built for local AI. Apple unveiled the new Mac Mini with M6 their first 2nm chip. Starting at $899. What you can run on the $899 Mac Mini (32GB) 👇 → Qwen 3.6 27B : fits comfortably, near-GPT-4 quality → Qwen 3 Coder 32B : full coding assistant on your desk → Llama 4 8B at full precision → DeepSeek R1 32B for reasoning tasks → 13.5x faster LLM processing than M1. 4x faster than M4. What you can run on the Mac Studio with M5 Ultra (512GB) 👇 → Llama 4 70B at high quality quantization → DeepSeek V4 with hundreds of billions of parameters → Frontier-scale models entirely in local memory → Connect four Mac Studios over Thunderbolt 5 cluster them to run trillion-parameter models locally Apple said their Mac business grew 30% last quarter largely because people are buying these to run AI models on their own desks instead of paying for cloud tokens. The future of inference is local. Apple built the hardware for it.
0:38

Apple is no longer hiding it. These machines are built for local AI. Apple unveiled the new Mac Mini with M6 their first 2nm chip. Starting at $899. What you can run on the $899 Mac Mini (32GB) 👇 → Qwen 3.6 27B : fits comfortably, near-GPT-4 quality → Qwen 3 Coder 32B : full coding assistant on your desk → Llama 4 8B at full precision → DeepSeek R1 32B for reasoning tasks → 13.5x faster LLM processing than M1. 4x faster than M4. What you can run on the Mac Studio with M5 Ultra (512GB) 👇 → Llama 4 70B at high quality quantization → DeepSeek V4 with hundreds of billions of parameters → Frontier-scale models entirely in local memory → Connect four Mac Studios over Thunderbolt 5 cluster them to run trillion-parameter models locally Apple said their Mac business grew 30% last quarter largely because people are buying these to run AI models on their own desks instead of paying for cloud tokens. The future of inference is local. Apple built the hardware for it.

Vaibhav Sisinty

518,197 次观看 • 2 个月前

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵
1:39

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

thestreamingdev()

733,913 次观看 • 7 个月前

⚡️ JUST IN: MUSK LAUNCHES GROK 4.5, CALLING IT "OPUS-CLASS" AI MODEL Musk just released Grok 4.5 on Grok Build, Cursor, and the SpaceXAI Console, calling it an "Opus-class" AI model. He says it is faster, more token-efficient and lower cost, with EU access expected by mid-July. Built on a 1.5 TRILLION parameter foundation model, it drops the same day as OpenAI's GPT-5.6. API pricing is $2 per million input tokens and $6 per million output tokens.
0:55

⚡️ JUST IN: MUSK LAUNCHES GROK 4.5, CALLING IT "OPUS-CLASS" AI MODEL Musk just released Grok 4.5 on Grok Build, Cursor, and the SpaceXAI Console, calling it an "Opus-class" AI model. He says it is faster, more token-efficient and lower cost, with EU access expected by mid-July. Built on a 1.5 TRILLION parameter foundation model, it drops the same day as OpenAI's GPT-5.6. API pricing is $2 per million input tokens and $6 per million output tokens.

Coin Bureau

50,488 次观看 • 4 个月前

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks
34:56

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks

Noisy

21,553 次观看 • 4 个月前

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓
1:21

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓

slash1s

101,317 次观看 • 4 个月前

🚨 2.8 trillion parameters. Under 4GB VRAM. No quantization. An open source tool called AirLLM has changed how large language models run on consumer hardware. Instead of loading the entire model at once, it processes one layer at a time. This means your VRAM needs depend only on the size of a single layer rather than the full model. No quantization. No distillation. No pruning. → DeepSeek V3 (671B) runs on just 12GB → Llama 3.1 405B runs on 8GB → Kimi K3 (2.8 trillion parameters) runs on under 4GB → Works with nearly every major open source model The largest model often needs the least VRAM. Kimi K3 is a sparse mixture of experts model, so it only loads the specific experts activated by each token instead of an entire dense layer. This is why a 2.8 trillion parameter model can run in less memory than a 70B model. AirLLM makes frontier scale AI practical on everyday hardware without any of the usual tradeoffs. (Link in the comments)
0:37

🚨 2.8 trillion parameters. Under 4GB VRAM. No quantization. An open source tool called AirLLM has changed how large language models run on consumer hardware. Instead of loading the entire model at once, it processes one layer at a time. This means your VRAM needs depend only on the size of a single layer rather than the full model. No quantization. No distillation. No pruning. → DeepSeek V3 (671B) runs on just 12GB → Llama 3.1 405B runs on 8GB → Kimi K3 (2.8 trillion parameters) runs on under 4GB → Works with nearly every major open source model The largest model often needs the least VRAM. Kimi K3 is a sparse mixture of experts model, so it only loads the specific experts activated by each token instead of an entire dense layer. This is why a 2.8 trillion parameter model can run in less memory than a 70B model. AirLLM makes frontier scale AI practical on everyday hardware without any of the usual tradeoffs. (Link in the comments)

Brooks Whale X 🐋

40,529 次观看 • 23 天前

“AI COULD SOON THINK 100× TO 1,000× FASTER THAN HUMANS.” Emad Mostaque says AI systems are about to become dramatically faster as specialized AI hardware improves. He says humans currently operate at roughly 20–30 tokens per second, while current AI models are around 40 tokens per second. He points to Taalas' technology, which he says demonstrated an 8-billion-parameter model generating around 14,000 tokens per second. Mostaque says current AI systems are effectively “thinking at the speed of a human”, but that will soon change. He predicts AI systems could reach 1,000 tokens per second, followed by 10,000 tokens per second by the end of next year. His broader point is that AI could eventually “think a thousand times faster” than humans.
1:58

“AI COULD SOON THINK 100× TO 1,000× FASTER THAN HUMANS.” Emad Mostaque says AI systems are about to become dramatically faster as specialized AI hardware improves. He says humans currently operate at roughly 20–30 tokens per second, while current AI models are around 40 tokens per second. He points to Taalas' technology, which he says demonstrated an 8-billion-parameter model generating around 14,000 tokens per second. Mostaque says current AI systems are effectively “thinking at the speed of a human”, but that will soon change. He predicts AI systems could reach 1,000 tokens per second, followed by 10,000 tokens per second by the end of next year. His broader point is that AI could eventually “think a thousand times faster” than humans.

SciTech Era

12,824 次观看 • 2 个月前

APPLE BUILT A $10,000 COMPUTER THAT RUNS WHAT USED TO REQUIRE AN ENTIRE SERVER ROOM 06:40 he shows the number that breaks everything - one Mac Studio is more powerful than four Dell or AMD machines combined and uses half the energy DeepSeek R1 doesn't run on even two competitors - you need four - one Mac Studio does it alone runs quieter than a fan in the room and uses less energy than a kettle in the kitchen four Mac Studios on a desk connected through a regular cable run a 1 trillion parameter model - what used to require an entire data center anyone can put this at home and Jeff Geerling proved it - run the same models that big companies use without cloud, without subscriptions, without data leaving the room
14:24

APPLE BUILT A $10,000 COMPUTER THAT RUNS WHAT USED TO REQUIRE AN ENTIRE SERVER ROOM 06:40 he shows the number that breaks everything - one Mac Studio is more powerful than four Dell or AMD machines combined and uses half the energy DeepSeek R1 doesn't run on even two competitors - you need four - one Mac Studio does it alone runs quieter than a fan in the room and uses less energy than a kettle in the kitchen four Mac Studios on a desk connected through a regular cable run a 1 trillion parameter model - what used to require an entire data center anyone can put this at home and Jeff Geerling proved it - run the same models that big companies use without cloud, without subscriptions, without data leaving the room

Noisy

26,305 次观看 • 4 个月前

4 MAC MINIS RUNNING A 671B PARAMETER MODEL AS A CLUSTER No data center, no Cloud, no expensive hardware and not a single API call.. Just 4 Mac minis connected through EXO running DeepSeek v3.1 671b locally and actually fast. The part nobody talks about is that you don’t need one monster machine, you can cluster old computers you already have and split the load between them. The full breakdown of tools that make this possible is in the article below. Save and Read it today ↓
0:29

4 MAC MINIS RUNNING A 671B PARAMETER MODEL AS A CLUSTER No data center, no Cloud, no expensive hardware and not a single API call.. Just 4 Mac minis connected through EXO running DeepSeek v3.1 671b locally and actually fast. The part nobody talks about is that you don’t need one monster machine, you can cluster old computers you already have and split the load between them. The full breakdown of tools that make this possible is in the article below. Save and Read it today ↓

slash1s

58,724 次观看 • 5 个月前

Fast tokens are the most valuable tokens. Because inference speed is productivity. And Cerebras makes the fastest tokens in the world. Google DeepMind's Gemma 4 runs at 1,500+ tokens per second on Cerebras — more than an order of magnitude faster than on GPUs and 15x faster than Claude Haiku 4.5. Private preview today. GA later this month. Build blazing fast multimodal apps.
0:36

Fast tokens are the most valuable tokens. Because inference speed is productivity. And Cerebras makes the fastest tokens in the world. Google DeepMind's Gemma 4 runs at 1,500+ tokens per second on Cerebras — more than an order of magnitude faster than on GPUs and 15x faster than Claude Haiku 4.5. Private preview today. GA later this month. Build blazing fast multimodal apps.

Andrew Feldman

12,008 次观看 • 4 个月前

The new 1 Trillion parameter Kimi K2 Thinking model runs well on 2 M3 Ultras in its native format - no loss in quality! The model was quantization aware trained (qat) at int4. Here it generated ~3500 tokens at 15 toks/sec using pipeline-parallelism in mlx-lm:
0:41

The new 1 Trillion parameter Kimi K2 Thinking model runs well on 2 M3 Ultras in its native format - no loss in quality! The model was quantization aware trained (qat) at int4. Here it generated ~3500 tokens at 15 toks/sec using pipeline-parallelism in mlx-lm:

Awni Hannun

501,223 次观看 • 1 年前

ONE BUILDER WIRED 4 RASPBERRY PI MODULES INTO A €450 AI CLUSTER WITH 64GB OF RAM AND KILLED HIS $200/MONTH CHATGPT SUBSCRIPTION 00:37 he points at the terminal, "loaded, loaded, loaded - 4 nodes synced over gigabit, 64 gigabytes total, llama 3.1 partitioned across the cluster" he stacked 4 raspberry pi CM5 lite modules with 16 gigabytes each onto a sipeed nanocluster board, the whole rig runs off a single 65 watt power supply and a gigabit internal switch distributed llama partitions the model across the nodes with synchronized workloads handled over the internal fabric, no quantization needed, the cluster hits about 30 tokens per second on small models a heavy developer pays $200 a month for chatgpt pro and another $200 for claude code, this rig cost €450 in parts and breaks even on the stack in month 2 most people will keep paying anthropic and openai forever, a few will spend a weekend wiring 4 raspberry pi modules and never see a subscription invoice again the window is open, follow and bookmark before it closes
0:37

ONE BUILDER WIRED 4 RASPBERRY PI MODULES INTO A €450 AI CLUSTER WITH 64GB OF RAM AND KILLED HIS $200/MONTH CHATGPT SUBSCRIPTION 00:37 he points at the terminal, "loaded, loaded, loaded - 4 nodes synced over gigabit, 64 gigabytes total, llama 3.1 partitioned across the cluster" he stacked 4 raspberry pi CM5 lite modules with 16 gigabytes each onto a sipeed nanocluster board, the whole rig runs off a single 65 watt power supply and a gigabit internal switch distributed llama partitions the model across the nodes with synchronized workloads handled over the internal fabric, no quantization needed, the cluster hits about 30 tokens per second on small models a heavy developer pays $200 a month for chatgpt pro and another $200 for claude code, this rig cost €450 in parts and breaks even on the stack in month 2 most people will keep paying anthropic and openai forever, a few will spend a weekend wiring 4 raspberry pi modules and never see a subscription invoice again the window is open, follow and bookmark before it closes

Antid

64,291 次观看 • 4 个月前

Live on Cline (v3.67.1) OpenAI 's GPT 5.3 Codex. The speed and token efficiency improvements are real. Here is what's new: > 25% faster than 5.2 Codex > #1 on SWE-Bench Pro (4 different programming languages) > Fewer tokens per task than any prior OpenAI model Runs cost less and finish faster. You can use it in Cline right now without setting up an API key. Select the model and try it on your repo.
0:31

Live on Cline (v3.67.1) OpenAI 's GPT 5.3 Codex. The speed and token efficiency improvements are real. Here is what's new: > 25% faster than 5.2 Codex > #1 on SWE-Bench Pro (4 different programming languages) > Fewer tokens per task than any prior OpenAI model Runs cost less and finish faster. You can use it in Cline right now without setting up an API key. Select the model and try it on your repo.

Cline

21,291 次观看 • 8 个月前

Jensen Huang said Grok 5 will be 7 Trillion parameter model. On time to train, with the training window fixed at 1 month, the new Vera-Rubin GPU system of Nvidia needs 1/4 the number of systems compared with Blackwell to train the same frontier model. “factory throughput” improves by about 10x over Blackwell, and Blackwell itself was about 10x over Hopper. i.e. overall Rubin is roughly 100x Hopper in factory throughput per watt, which matters hugely, because a 1 GW, $50B data center is power-limited and revenue scales with throughput per watt. --- From Nvidia YT channel
1:24

Jensen Huang said Grok 5 will be 7 Trillion parameter model. On time to train, with the training window fixed at 1 month, the new Vera-Rubin GPU system of Nvidia needs 1/4 the number of systems compared with Blackwell to train the same frontier model. “factory throughput” improves by about 10x over Blackwell, and Blackwell itself was about 10x over Hopper. i.e. overall Rubin is roughly 100x Hopper in factory throughput per watt, which matters hugely, because a 1 GW, $50B data center is power-limited and revenue scales with throughput per watt. --- From Nvidia YT channel

Rohan Paul

165,770 次观看 • 10 个月前

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).
0:50

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).

Bin Lin

27,792 次观看 • 8 个月前