Loading video...

Video Failed to Load

Go Home

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's...

noisyb0y1's profile picture

Noisy

24,941 subscribers

48,615 views • 2 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

APPLE BUILT A $10,000 COMPUTER THAT RUNS WHAT USED TO REQUIRE AN ENTIRE SERVER ROOM 06:40 he shows the number that breaks everything - one Mac Studio is more powerful than four Dell or AMD machines combined and uses half the energy DeepSeek R1 doesn't run on even two competitors - you need four - one Mac Studio does it alone runs quieter than a fan in the room and uses less energy than a kettle in the kitchen four Mac Studios on a desk connected through a regular cable run a 1 trillion parameter model - what used to require an entire data center anyone can put this at home and Jeff Geerling proved it - run the same models that big companies use without cloud, without subscriptions, without data leaving the room
14:24

APPLE BUILT A $10,000 COMPUTER THAT RUNS WHAT USED TO REQUIRE AN ENTIRE SERVER ROOM 06:40 he shows the number that breaks everything - one Mac Studio is more powerful than four Dell or AMD machines combined and uses half the energy DeepSeek R1 doesn't run on even two competitors - you need four - one Mac Studio does it alone runs quieter than a fan in the room and uses less energy than a kettle in the kitchen four Mac Studios on a desk connected through a regular cable run a 1 trillion parameter model - what used to require an entire data center anyone can put this at home and Jeff Geerling proved it - run the same models that big companies use without cloud, without subscriptions, without data leaving the room

Noisy

26,305 views • 2 months ago

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours
22:35

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

Noisy

79,665 views • 2 months ago

THIS CLUSTER OF 4 MAC STUDIOS USES LESS POWER THAN ONE NVIDIA CHIP - AND COSTS 5X LESS 17:30 the number hits hard - 600 watts for all four Mac Studios - less than one H200 that costs $30,000 on its own instead of one $30,000 GPU you get 4 machines with 1.5TB of memory, 3x more performance and models that simply cannot run on a single chip Apple quietly added RDMA over Thunderbolt in a beta update - latency dropped 99% and the same cluster became 3x faster without buying any new hardware before the update: 5 tokens per second. after: 15.5 - a free upgrade through a software update 8 Nvidia DGX Sparks for the same memory would cost $32,000 - this cluster costs a fraction and gets faster with every update 10x more productive at half the price - and the only thing you need is knowing where to look
20:03

THIS CLUSTER OF 4 MAC STUDIOS USES LESS POWER THAN ONE NVIDIA CHIP - AND COSTS 5X LESS 17:30 the number hits hard - 600 watts for all four Mac Studios - less than one H200 that costs $30,000 on its own instead of one $30,000 GPU you get 4 machines with 1.5TB of memory, 3x more performance and models that simply cannot run on a single chip Apple quietly added RDMA over Thunderbolt in a beta update - latency dropped 99% and the same cluster became 3x faster without buying any new hardware before the update: 5 tokens per second. after: 15.5 - a free upgrade through a software update 8 Nvidia DGX Sparks for the same memory would cost $32,000 - this cluster costs a fraction and gets faster with every update 10x more productive at half the price - and the only thing you need is knowing where to look

Noisy

106,054 views • 2 months ago

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓
1:21

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓

slash1s

101,317 views • 2 months ago

Apple is no longer hiding it. These machines are built for local AI. Apple unveiled the new Mac Mini with M6 their first 2nm chip. Starting at $899. What you can run on the $899 Mac Mini (32GB) 👇 → Qwen 3.6 27B : fits comfortably, near-GPT-4 quality → Qwen 3 Coder 32B : full coding assistant on your desk → Llama 4 8B at full precision → DeepSeek R1 32B for reasoning tasks → 13.5x faster LLM processing than M1. 4x faster than M4. What you can run on the Mac Studio with M5 Ultra (512GB) 👇 → Llama 4 70B at high quality quantization → DeepSeek V4 with hundreds of billions of parameters → Frontier-scale models entirely in local memory → Connect four Mac Studios over Thunderbolt 5 cluster them to run trillion-parameter models locally Apple said their Mac business grew 30% last quarter largely because people are buying these to run AI models on their own desks instead of paying for cloud tokens. The future of inference is local. Apple built the hardware for it.
0:38

Apple is no longer hiding it. These machines are built for local AI. Apple unveiled the new Mac Mini with M6 their first 2nm chip. Starting at $899. What you can run on the $899 Mac Mini (32GB) 👇 → Qwen 3.6 27B : fits comfortably, near-GPT-4 quality → Qwen 3 Coder 32B : full coding assistant on your desk → Llama 4 8B at full precision → DeepSeek R1 32B for reasoning tasks → 13.5x faster LLM processing than M1. 4x faster than M4. What you can run on the Mac Studio with M5 Ultra (512GB) 👇 → Llama 4 70B at high quality quantization → DeepSeek V4 with hundreds of billions of parameters → Frontier-scale models entirely in local memory → Connect four Mac Studios over Thunderbolt 5 cluster them to run trillion-parameter models locally Apple said their Mac business grew 30% last quarter largely because people are buying these to run AI models on their own desks instead of paying for cloud tokens. The future of inference is local. Apple built the hardware for it.

Vaibhav Sisinty

504,858 views • 4 days ago

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.
2:41

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Milk Road AI

37,459 views • 1 month ago

4 MAC MINIS RUNNING A 671B PARAMETER MODEL AS A CLUSTER No data center, no Cloud, no expensive hardware and not a single API call.. Just 4 Mac minis connected through EXO running DeepSeek v3.1 671b locally and actually fast. The part nobody talks about is that you don’t need one monster machine, you can cluster old computers you already have and split the load between them. The full breakdown of tools that make this possible is in the article below. Save and Read it today ↓
0:29

4 MAC MINIS RUNNING A 671B PARAMETER MODEL AS A CLUSTER No data center, no Cloud, no expensive hardware and not a single API call.. Just 4 Mac minis connected through EXO running DeepSeek v3.1 671b locally and actually fast. The part nobody talks about is that you don’t need one monster machine, you can cluster old computers you already have and split the load between them. The full breakdown of tools that make this possible is in the article below. Save and Read it today ↓

slash1s

58,724 views • 2 months ago

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks
34:56

THIS DEVELOPER RAN THE LARGEST AI MODEL IN THE WORLD ON 5 MAC STUDIOS - AND IT COST 100X LESS THAN WHAT OPENAI USES 27:47 he says it after hours of setup - Llama 3.1 405B running locally on five Mac Studios - a model that normally requires 42 GPUs at $1,600 each or an entire data center 320GB unified memory combined - every Mac Studio adds its 64GB into one shared pool - and a model that needs 1TB of VRAM just distributes across all five it launched, it responded and not a single token went to OpenAI or Google - everything local, everything private, everything yours and 100x cheaper 5 Mac Studios, 46 watts for all of them in idle - less than one light bulb per machine what OpenAI keeps behind a subscription he ran at home with a one-time purchase in a few clicks

Noisy

21,553 views • 2 months ago

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less
24:17

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less

Noisy

185,632 views • 2 months ago

Not every AI interaction is best served with a large frontier model. There is a long tail of trillion-token use cases, from tagging to search, best served by a sub-10b-parameter model that runs in milliseconds and costs many orders of magnitude less than the frontier. We built Freesolo Flash to make training loops like SFT and RL easy and end-to-end completable through your coding agent.
2:02

Not every AI interaction is best served with a large frontier model. There is a long tail of trillion-token use cases, from tagging to search, best served by a sub-10b-parameter model that runs in milliseconds and costs many orders of magnitude less than the frontier. We built Freesolo Flash to make training loops like SFT and RL easy and end-to-end completable through your coding agent.

Freesolo

28,456 views • 1 month ago

APPLE JUST SHOWED HOW TO RUN 10 AI AGENTS LOCALLY ON MAC - NO CLOUD, NO API KEYS, ZERO COSTS 00:10 Apple engineer says: "your data stays on your device, AI available anywhere at any time, zero usage costs" agent reads your code, checks GitHub, finds what needs attention and writes a report - all on your Mac, nothing goes to the internet 10 agents work simultaneously - one writes code, another tests, third fixes bugs - parallel with no queue built a full iPad app from scratch in 2 minutes, fixed its own mistakes and compiled without issues takes 5 minutes to set up - and you never pay again for a bot that runs 24/7
13:29

APPLE JUST SHOWED HOW TO RUN 10 AI AGENTS LOCALLY ON MAC - NO CLOUD, NO API KEYS, ZERO COSTS 00:10 Apple engineer says: "your data stays on your device, AI available anywhere at any time, zero usage costs" agent reads your code, checks GitHub, finds what needs attention and writes a report - all on your Mac, nothing goes to the internet 10 agents work simultaneously - one writes code, another tests, third fixes bugs - parallel with no queue built a full iPad app from scratch in 2 minutes, fixed its own mistakes and compiled without issues takes 5 minutes to set up - and you never pay again for a bot that runs 24/7

Noisy

525,385 views • 2 months ago

Google just shipped a free dictation app for Mac and iPhone called AI Edge Eloquent, and the model running it is Gemma 4 12B, entirely on your device. What it actually does: you speak, it transcribes, removes filler words, and polishes the result before dropping clean text into whatever app you are using. You can also give it voice commands to reformat, like "translate to Hindi" or "make this a bullet list." Audio and video file transcription is built in too. The interesting part is the privacy angle. Most AI writing tools send your audio to a server. This one never leaves your machine, which matters if you are dictating anything sensitive. The tradeoff is hardware: you need a Mac with at least 16GB of RAM to run a 12B parameter model comfortably.
1:15

Google just shipped a free dictation app for Mac and iPhone called AI Edge Eloquent, and the model running it is Gemma 4 12B, entirely on your device. What it actually does: you speak, it transcribes, removes filler words, and polishes the result before dropping clean text into whatever app you are using. You can also give it voice commands to reformat, like "translate to Hindi" or "make this a bullet list." Audio and video file transcription is built in too. The interesting part is the privacy angle. Most AI writing tools send your audio to a server. This one never leaves your machine, which matters if you are dictating anything sensitive. The tradeoff is hardware: you need a Mac with at least 16GB of RAM to run a 12B parameter model comfortably.

Aditya ⚡Rao

85,060 views • 2 months ago

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.
2:00

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.

Aakash Gupta

508,446 views • 4 months ago

You don't need a trillion-parameter model that understands physics to answer a customer support call. But that's not even the main reason we build small models. The real one: LLMs store everything. Literally everything all of it sitting in the weights. Humans don't work that way, and your agent shouldn't either. Full podcast, link in thread.
1:03

You don't need a trillion-parameter model that understands physics to answer a customer support call. But that's not even the main reason we build small models. The real one: LLMs store everything. Literally everything all of it sitting in the weights. Humans don't work that way, and your agent shouldn't either. Full podcast, link in thread.

Sudarshan Kamath

19,950 views • 1 month ago

CHINESE BUSINESSMAN BOUGHT 4 MAC MINIS FOR $1,699 AND CONNECTED THEM TO HIS MACBOOK - NOW THE CORNER OF HIS DESK MAKES MORE THAN MOST OFFICES MacBook Pro 16" runs everything - assigns tasks, controls the flow, holds context - 4 Mac Minis in the corner execute inference in parallel and don't know what downtime means he connected them through EXO - an open source framework that combines multiple Apple Silicon devices into one unified cluster - and together they run models no single device could handle alone while everyone else rents cloud GPUs for $1,900/month he paid $1,699 once and the entire cluster now runs on $20/month in electricity and nothing else clients pay for inference, data never leaves the room and the cluster scales by simply placing one more Mac Mini on top the MacBook is what he works on - the Mac Minis are what work for him around the clock without days off $1,699 invested once - and a corner of a desk that makes more than most companies with a rented office
0:20

CHINESE BUSINESSMAN BOUGHT 4 MAC MINIS FOR $1,699 AND CONNECTED THEM TO HIS MACBOOK - NOW THE CORNER OF HIS DESK MAKES MORE THAN MOST OFFICES MacBook Pro 16" runs everything - assigns tasks, controls the flow, holds context - 4 Mac Minis in the corner execute inference in parallel and don't know what downtime means he connected them through EXO - an open source framework that combines multiple Apple Silicon devices into one unified cluster - and together they run models no single device could handle alone while everyone else rents cloud GPUs for $1,900/month he paid $1,699 once and the entire cluster now runs on $20/month in electricity and nothing else clients pay for inference, data never leaves the room and the cluster scales by simply placing one more Mac Mini on top the MacBook is what he works on - the Mac Minis are what work for him around the clock without days off $1,699 invested once - and a corner of a desk that makes more than most companies with a rented office

Noisy

22,391 views • 2 months ago

This guy plugged a DGX Spark (the $3K Nvidia box) and a Mac Mini M4 together to run AI and what happened next surprised everyone > the Nvidia box handles the hard part - processing your prompt in milliseconds > Mac Mini M4 handles the fast part - generating the response at memory bandwidth speeds nothing else can match > together they hit 84 tokens per second on Llama - 6x faster than the Spark alone > running compute agents locally on this setup means your data never leaves your hardware > two boxes. two different architectures. one AI system that Deepseek runs at data center scale he ran it on his desk save this. the way we build local AI is about to change
18:41

This guy plugged a DGX Spark (the $3K Nvidia box) and a Mac Mini M4 together to run AI and what happened next surprised everyone > the Nvidia box handles the hard part - processing your prompt in milliseconds > Mac Mini M4 handles the fast part - generating the response at memory bandwidth speeds nothing else can match > together they hit 84 tokens per second on Llama - 6x faster than the Spark alone > running compute agents locally on this setup means your data never leaves your hardware > two boxes. two different architectures. one AI system that Deepseek runs at data center scale he ran it on his desk save this. the way we build local AI is about to change

Mr. Buzzoni

153,768 views • 3 months ago

INTEL JUST SHIPPED A WORKSTATION CARD WITH TWO GPUs ON ONE PCB AND 48GB OF VRAM. FOUR CARDS GIVE A SINGLE MOTHERBOARD 192GB OF POOLED INFERENCE MEMORY FOR THE PRICE OF TWO RTX 5090s 00:14 he holds one card up to the camera, two GPU dies side by side under the cooler, each one running its own x8 PCIe lane back to the chipset the cluster sees eight discrete accelerators in software. intel's Battle Matrix stack shards a model across all eight, so a 235B parameter network loads in slices and answers requests in parallel what 192GB of VRAM unlocks: an entire 200B class model in memory without quantization. a vision agent reading 100 invoices at once. a research box that holds three frontier models loaded simultaneously, switching between them in under a second intel is the slow side of inference. nvidia is faster per token, that is the honest tradeoff. but the only other path to this much VRAM is a $40,000 nvidia rack or three networked Mac Studios four B60 cards plus the chassis lands under $5,000. power draw averages 800 watts, $55 a month in electricity. one engineer paying $400 a month for combined ChatGPT Pro and Claude Code Max pays the hardware off in less than a year
1:16

INTEL JUST SHIPPED A WORKSTATION CARD WITH TWO GPUs ON ONE PCB AND 48GB OF VRAM. FOUR CARDS GIVE A SINGLE MOTHERBOARD 192GB OF POOLED INFERENCE MEMORY FOR THE PRICE OF TWO RTX 5090s 00:14 he holds one card up to the camera, two GPU dies side by side under the cooler, each one running its own x8 PCIe lane back to the chipset the cluster sees eight discrete accelerators in software. intel's Battle Matrix stack shards a model across all eight, so a 235B parameter network loads in slices and answers requests in parallel what 192GB of VRAM unlocks: an entire 200B class model in memory without quantization. a vision agent reading 100 invoices at once. a research box that holds three frontier models loaded simultaneously, switching between them in under a second intel is the slow side of inference. nvidia is faster per token, that is the honest tradeoff. but the only other path to this much VRAM is a $40,000 nvidia rack or three networked Mac Studios four B60 cards plus the chassis lands under $5,000. power draw averages 800 watts, $55 a month in electricity. one engineer paying $400 a month for combined ChatGPT Pro and Claude Code Max pays the hardware off in less than a year

NO1ennn

48,247 views • 2 months ago

THE $599 MAC MINI GETS A $49 DOCK AND TURNS FROM A DESK COMPUTER INTO A TINY 24/7 AI WORKFLOW SERVER 00:19 he clips the dock onto the back of the mac mini, adds extra ports, ethernet, storage and turns one small silver box into a workflow hub that can stay online for 700+ hours a month a base mac mini can sit online all day, pulling light power while scripts watch folders, move files, process notes and keep boring workflows running in the background paired with claude, it becomes a small home automation box: 2-hour lectures get transcribed, 30-page articles get summarized, reports get drafted, and research gets cleaned before you even open the laptop the difference is the loop. trigger, process, verify, save, repeat. the same workflow that costs $40-100/month across cloud tools can run locally from one machine on your desk that is why this tiny upgrade matters. the dock is cheap, but it gives the mac mini enough expansion to act like a quiet server instead of another computer waiting for you to use it bookmark this before your desk setup realizes it became infrastructure.
0:29

THE $599 MAC MINI GETS A $49 DOCK AND TURNS FROM A DESK COMPUTER INTO A TINY 24/7 AI WORKFLOW SERVER 00:19 he clips the dock onto the back of the mac mini, adds extra ports, ethernet, storage and turns one small silver box into a workflow hub that can stay online for 700+ hours a month a base mac mini can sit online all day, pulling light power while scripts watch folders, move files, process notes and keep boring workflows running in the background paired with claude, it becomes a small home automation box: 2-hour lectures get transcribed, 30-page articles get summarized, reports get drafted, and research gets cleaned before you even open the laptop the difference is the loop. trigger, process, verify, save, repeat. the same workflow that costs $40-100/month across cloud tools can run locally from one machine on your desk that is why this tiny upgrade matters. the dock is cheap, but it gives the mac mini enough expansion to act like a quiet server instead of another computer waiting for you to use it bookmark this before your desk setup realizes it became infrastructure.

Gipp 🦅

73,740 views • 2 months ago

Andrej Karpathy just made one of the most interesting arguments about AI model design that most people are completely missing. His take is that frontier AI models are not too big because the technology is complex and too big because the training data is garbage. When you or I think of the internet, we picture Wall Street Journal articles, Wikipedia entries, serious writing. That is not what a pretraining dataset looks like. When researchers at frontier labs look at random documents from the actual training corpus, it is stock ticker symbols, broken HTML, spam, gibberish. One estimate puts Llama 3's information compression at just 0.07 bits per token meaning the model has only a hazy recollection of most of what it trained on. So we build trillion parameter models not because we need a trillion parameter brain but because we need a trillion-parameter compression engine to squeeze some intelligence out of a firehose of noise. Most of those parameters are doing memory work, not cognitive work. Karpathy's prediction is separate the two entirely. Build a cognitive core, a model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization and pair it with external memory that it can query when it needs facts. He thinks a cognitive core trained on high-quality data could hit genuine intelligence at around one billion parameters. For reference, today's flagship models run between 200 billion and 1.8 trillion parameters with most of that weight dedicated to remembering the internet's slop. The trend is already moving his direction. GPT-4o operates at roughly 200 billion parameters and outperforms the original 1.8 trillion-parameter GPT-4. Inference costs for GPT-3.5-level performance dropped 280-fold between 2022 and 2024 driven almost entirely by smaller, cleaner, better-architected models. The real bottleneck in AI right now is not compute but rather data quality.
2:00

Andrej Karpathy just made one of the most interesting arguments about AI model design that most people are completely missing. His take is that frontier AI models are not too big because the technology is complex and too big because the training data is garbage. When you or I think of the internet, we picture Wall Street Journal articles, Wikipedia entries, serious writing. That is not what a pretraining dataset looks like. When researchers at frontier labs look at random documents from the actual training corpus, it is stock ticker symbols, broken HTML, spam, gibberish. One estimate puts Llama 3's information compression at just 0.07 bits per token meaning the model has only a hazy recollection of most of what it trained on. So we build trillion parameter models not because we need a trillion parameter brain but because we need a trillion-parameter compression engine to squeeze some intelligence out of a firehose of noise. Most of those parameters are doing memory work, not cognitive work. Karpathy's prediction is separate the two entirely. Build a cognitive core, a model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization and pair it with external memory that it can query when it needs facts. He thinks a cognitive core trained on high-quality data could hit genuine intelligence at around one billion parameters. For reference, today's flagship models run between 200 billion and 1.8 trillion parameters with most of that weight dedicated to remembering the internet's slop. The trend is already moving his direction. GPT-4o operates at roughly 200 billion parameters and outperforms the original 1.8 trillion-parameter GPT-4. Inference costs for GPT-3.5-level performance dropped 280-fold between 2022 and 2024 driven almost entirely by smaller, cleaner, better-architected models. The real bottleneck in AI right now is not compute but rather data quality.

Milk Road AI

200,545 views • 4 months ago

APPLE CHARGES $200 TO GO FROM 256GB TO 512GB ON THE MAC MINI. $400 FOR 1TB. $600 FOR 2TB. AND YOU HAVE TO DECIDE BEFORE YOU BUY. Pick wrong and you're stuck forever. Apple soldered the storage shut. That's their entire business model. Sell the box cheap, charge a premium for every upgrade you can't do yourself. One dock just broke the whole system. The Acasis 8-in-1 plugs into the Mac Mini through a single Thunderbolt cable. Slide any NVMe SSD into the top. 2,819 MB/s write, 2,857 MB/s read. Faster than Apple's own base storage. A 2TB NVMe SSD costs about $100 on Amazon. Apple charges $600 for the same upgrade. That's a 6x markup for storage you could add yourself in 10 seconds. But it doesn't stop at storage. SD card reader, DisplayPort, USB ports, power delivery. Every port Apple left out, this dock puts back in. Surface temp under load: 26.7°C. Same aluminum finish. Sits right next to the Mac Mini like Apple designed it. That's when it clicks. The $599 Mac Mini was never the cheap option. It was the entry fee. The real cost was every upgrade Apple locked behind the checkout page. One $150 dock and a $100 SSD just replaced $600 worth of Apple tax. Still think you need the upgraded model?
0:42

APPLE CHARGES $200 TO GO FROM 256GB TO 512GB ON THE MAC MINI. $400 FOR 1TB. $600 FOR 2TB. AND YOU HAVE TO DECIDE BEFORE YOU BUY. Pick wrong and you're stuck forever. Apple soldered the storage shut. That's their entire business model. Sell the box cheap, charge a premium for every upgrade you can't do yourself. One dock just broke the whole system. The Acasis 8-in-1 plugs into the Mac Mini through a single Thunderbolt cable. Slide any NVMe SSD into the top. 2,819 MB/s write, 2,857 MB/s read. Faster than Apple's own base storage. A 2TB NVMe SSD costs about $100 on Amazon. Apple charges $600 for the same upgrade. That's a 6x markup for storage you could add yourself in 10 seconds. But it doesn't stop at storage. SD card reader, DisplayPort, USB ports, power delivery. Every port Apple left out, this dock puts back in. Surface temp under load: 26.7°C. Same aluminum finish. Sits right next to the Mac Mini like Apple designed it. That's when it clicks. The $599 Mac Mini was never the cheap option. It was the entry fee. The real cost was every upgrade Apple locked behind the checkout page. One $150 dock and a $100 SSD just replaced $600 worth of Apple tax. Still think you need the upgraded model?

Framez

49,948 views • 19 days ago