KyzoroX's banner

KyzoroX

@kyzoroXX • 1,394 subscribers

AI • Automation • Innovation

Shorts

This is what happens when 48GB of VRAM isn't enough. Someone put 4× NVIDIA RTX A6000 inside an HP Z8 Fury G5. That gives you 192GB of VRAM across four GPUs. But here's the interesting part: You don't magically get one 192GB GPU. The memory is split across four cards, and the software has to manage how the model is distributed between them. That's why these machines are interesting for local AI. The problem isn't always compute. Sometimes you just need somewhere to put the model. Would you build this today, or go for a newer GPU with more bandwidth?

This is what happens when 48GB of VRAM isn't enough. Someone put 4× NVIDIA RTX A6000 inside an HP Z8 Fury G5. That gives you 192GB of VRAM across four GPUs. But here's the interesting part: You don't magically get one 192GB GPU. The memory is split across four cards, and the software has to manage how the model is distributed between them. That's why these machines are interesting for local AI. The problem isn't always compute. Sometimes you just need somewhere to put the model. Would you build this today, or go for a newer GPU with more bandwidth?

81,436 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

"Your Mac is useless for AI without CUDA, buy the $4K DGX." Half right — and the half that's wrong costs you $4,000. CUDA matters for training and some frameworks. Real. But "running local models"? I benchmarked a Mac Mini against the DGX on the same 30B: generation was 56 vs 84 t/s. Not useless — usable, faster than you read, no CUDA required. llama.cpp and Ollama don't care what logo is on the chip. Here's the honest split the video skips: - Commercial fine-tuning, CUDA-only pipelines, big-context prefill → yes, the DGX earns its price - Running and chatting with local models → your Mac already does this fine The DGX isn't overpriced. It's mis-pitched. It's a prefill-and-training machine, not a "your Mac is trash" machine. Buy it for the workload that needs it — not out of CUDA FOMO. Full 3-way benchmark (DGX vs Strix Halo vs Mac Mini, prefill vs generation) pinned.

264,048 просмотров • 3 месяцев назад

768GB of VRAM in one 4U server. That number sounds ridiculous until you remember it’s not one 768GB GPU. This Gigabyte G494-ZB4 packs: 8× RTX PRO 6000 Blackwell 768GB VRAM 2× EPYC 9554 1TB DDR5 ECC And that’s the part people often miss with multi-GPU systems. You can have almost 1TB of GPU memory, but the model still has to be split across 8 separate memory pools. So I think the next AI hardware race isn't just about how much memory you can put in a server. It's about how close that memory can behave to one giant pool. That’s a much harder problem to solve.

Sensitive content

This media may contain sensitive content.

768GB of VRAM in one 4U server. That number sounds ridiculous until you remember it’s not one 768GB GPU. This Gigabyte G494-ZB4 packs: 8× RTX PRO 6000 Blackwell 768GB VRAM 2× EPYC 9554 1TB DDR5 ECC And that’s the part people often miss with multi-GPU systems. You can have almost 1TB of GPU memory, but the model still has to be split across 8 separate memory pools. So I think the next AI hardware race isn't just about how much memory you can put in a server. It's about how close that memory can behave to one giant pool. That’s a much harder problem to solve.

88,405 просмотров • 1 месяц назад

For years, local AI had one answer: buy an NVIDIA GPU. RTX 3090. 24GB VRAM. RTX 4090. 24GB VRAM. RTX 5090. 32GB VRAM. And if you wanted to run something bigger, the price climbed fast. Then this video shows a different reality. An old M1 MacBook Air. 16GB unified memory. A DeepSeek R1 0528 Qwen3 8B model. Just 4.62GB on disk. Then the context gets pushed to almost 50,000 tokens. And the 16GB machine stays in the green. Meanwhile, the 8GB Mac is already sitting at 7.51GB used out of 8GB. That's the part I think people still underestimate. Local AI isn't one giant hardware market anymore. A $4,000 RTX 5090 setup, a 96GB Strix Halo box and an old M1 MacBook aren't competing for the same job. They're different rungs. The mistake is buying hardware for the model you wish you needed instead of the work you actually do. If an 8B model can handle your coding, documents, automation and daily tasks, why are you pricing a 32GB GPU? And that's where the subscription math gets interesting. A lot of people are still paying every month for AI without ever checking what the laptop already sitting on their desk can do. The best local AI machine might not be the fastest one. It might be the one you already own.

For years, local AI had one answer: buy an NVIDIA GPU. RTX 3090. 24GB VRAM. RTX 4090. 24GB VRAM. RTX 5090. 32GB VRAM. And if you wanted to run something bigger, the price climbed fast. Then this video shows a different reality. An old M1 MacBook Air. 16GB unified memory. A DeepSeek R1 0528 Qwen3 8B model. Just 4.62GB on disk. Then the context gets pushed to almost 50,000 tokens. And the 16GB machine stays in the green. Meanwhile, the 8GB Mac is already sitting at 7.51GB used out of 8GB. That's the part I think people still underestimate. Local AI isn't one giant hardware market anymore. A $4,000 RTX 5090 setup, a 96GB Strix Halo box and an old M1 MacBook aren't competing for the same job. They're different rungs. The mistake is buying hardware for the model you wish you needed instead of the work you actually do. If an 8B model can handle your coding, documents, automation and daily tasks, why are you pricing a 32GB GPU? And that's where the subscription math gets interesting. A lot of people are still paying every month for AI without ever checking what the laptop already sitting on their desk can do. The best local AI machine might not be the fastest one. It might be the one you already own.

67,357 просмотров • 2 месяцев назад

Someone put 1TB of RAM and 48GB of VRAM into a workstation just to run AI. This is an HP Z8 G4 with two 24-core Xeons, 96 threads, 1TB of RAM and an NVIDIA RTX A6000. On paper, it looks excessive. But for local AI, there’s a reason this kind of hardware still makes sense. The A6000 has 48GB of VRAM, while the system has another 1TB of RAM behind it. That gives you a lot more room for large models without immediately running into memory limits. The interesting part is that none of this is cutting-edge hardware. It’s an older workstation being reused for a workload that didn’t exist at this scale when the machine was designed. And that’s becoming one of the best things about Local AI: you don't always need new hardware. Sometimes you just need enough memory.

Someone put 1TB of RAM and 48GB of VRAM into a workstation just to run AI. This is an HP Z8 G4 with two 24-core Xeons, 96 threads, 1TB of RAM and an NVIDIA RTX A6000. On paper, it looks excessive. But for local AI, there’s a reason this kind of hardware still makes sense. The A6000 has 48GB of VRAM, while the system has another 1TB of RAM behind it. That gives you a lot more room for large models without immediately running into memory limits. The interesting part is that none of this is cutting-edge hardware. It’s an older workstation being reused for a workload that didn’t exist at this scale when the machine was designed. And that’s becoming one of the best things about Local AI: you don't always need new hardware. Sometimes you just need enough memory.

70,222 просмотров • 2 месяцев назад

We used to buy more VRAM. Now we’re finding ways around the VRAM limit. This little DDR5 SO-DIMM setup is paired with an old NVIDIA Tesla P40 in the video. The P40 is an old datacenter GPU with 24GB of VRAM. By today’s standards, it’s not fast. But combine that with a machine carrying 128GB of system memory, and suddenly you have a very different local AI box. The GPU handles what it’s good at. The system memory gives the model somewhere to go when VRAM runs out. You obviously pay for that with bandwidth and inference speed. But this is the part I find interesting: Local AI is making “how much memory do I have?” almost as important as “how powerful is my GPU?” We’ve spent years chasing faster GPUs. Maybe the next wave of local AI hardware will be about finding smarter ways to use all the memory we already have. More VRAM is still better. But it’s no longer the only way to run a bigger model.

We used to buy more VRAM. Now we’re finding ways around the VRAM limit. This little DDR5 SO-DIMM setup is paired with an old NVIDIA Tesla P40 in the video. The P40 is an old datacenter GPU with 24GB of VRAM. By today’s standards, it’s not fast. But combine that with a machine carrying 128GB of system memory, and suddenly you have a very different local AI box. The GPU handles what it’s good at. The system memory gives the model somewhere to go when VRAM runs out. You obviously pay for that with bandwidth and inference speed. But this is the part I find interesting: Local AI is making “how much memory do I have?” almost as important as “how powerful is my GPU?” We’ve spent years chasing faster GPUs. Maybe the next wave of local AI hardware will be about finding smarter ways to use all the memory we already have. More VRAM is still better. But it’s no longer the only way to run a bigger model.

35,554 просмотров • 1 месяц назад

$RTX 3090 was never designed to be a server GPU. Yet someone just built a 4-GPU AI workstation around four of them. 4× RTX 3090 96GB VRAM 256GB ECC RAM AMD EPYC 7K62 2000W PSU 11 fans The interesting part isn't the performance. It's the memory. Four 24GB cards can be found for a fraction of what you'd pay for modern datacenter GPUs with similar total VRAM. The catch is obvious: 96GB doesn't behave like one 96GB GPU. You still have to deal with GPU-to-GPU communication, power and heat. But for local AI, this is exactly why the 3090 refuses to disappear. Would you build this today?$

RTX 3090 was never designed to be a server GPU. Yet someone just built a 4-GPU AI workstation around four of them. 4× RTX 3090 96GB VRAM 256GB ECC RAM AMD EPYC 7K62 2000W PSU 11 fans The interesting part isn't the performance. It's the memory. Four 24GB cards can be found for a fraction of what you'd pay for modern datacenter GPUs with similar total VRAM. The catch is obvious: 96GB doesn't behave like one 96GB GPU. You still have to deal with GPU-to-GPU communication, power and heat. But for local AI, this is exactly why the 3090 refuses to disappear. Would you build this today?

36,552 просмотров • 1 месяц назад

$3,800 for a Mini PC with 96GB of GPU memory sounds ridiculous. Until you see what AMD is actually doing. The GMKtec EVO-X3 has: Ryzen AI MAX+ 395 128GB LPDDR5X Up to 96GB allocated to the Radeon 8060S 8000 MT/s memory No discrete GPU. That means a tiny box can run local models that would normally push you toward a workstation with a huge GPU. And it can still play games. The interesting part isn't the 126 TOPS number. It's the fact that 96GB of GPU memory now fits inside a mini PC. The question is no longer whether this can run a big model. It's how fast it can run it. The EVO-X3 is available here:

$3,800 for a Mini PC with 96GB of GPU memory sounds ridiculous. Until you see what AMD is actually doing. The GMKtec EVO-X3 has: Ryzen AI MAX+ 395 128GB LPDDR5X Up to 96GB allocated to the Radeon 8060S 8000 MT/s memory No discrete GPU. That means a tiny box can run local models that would normally push you toward a workstation with a huge GPU. And it can still play games. The interesting part isn't the 126 TOPS number. It's the fact that 96GB of GPU memory now fits inside a mini PC. The question is no longer whether this can run a big model. It's how fast it can run it. The EVO-X3 is available here:

24,654 просмотров • 1 месяц назад

Someone built a workstation that needs 4.5 kilowatts. That's more than any standard wall circuit can deliver. Renderboxes Molecule Air: eight RTX 6000 Ada, a Threadripper PRO 7975WX, 768GB of DDR5 ECC 5800, and three Corsair HX1500i supplies stacked together. 🔴 US 15A circuit → 1,800W 🔴 US 20A circuit → 2,400W 🔴 EU 16A circuit → 3,680W 🟢 This machine → 4,500W The PSU rating isn't the draw, though. Eight cards at 300W plus the CPU lands near 3kW sustained. That's three space heaters running in the room, and the cooling to remove that heat costs more power on top. Note: 384GB of VRAM total, but it's eight separate 48GB pools talking over PCIe. Not one pool of 384. This is what "local AI" looks like at the top end. A datacenter rack that happens to be in a room. So before the GPU budget — what does your panel actually deliver?

Someone built a workstation that needs 4.5 kilowatts. That's more than any standard wall circuit can deliver. Renderboxes Molecule Air: eight RTX 6000 Ada, a Threadripper PRO 7975WX, 768GB of DDR5 ECC 5800, and three Corsair HX1500i supplies stacked together. 🔴 US 15A circuit → 1,800W 🔴 US 20A circuit → 2,400W 🔴 EU 16A circuit → 3,680W 🟢 This machine → 4,500W The PSU rating isn't the draw, though. Eight cards at 300W plus the CPU lands near 3kW sustained. That's three space heaters running in the room, and the cooling to remove that heat costs more power on top. Note: 384GB of VRAM total, but it's eight separate 48GB pools talking over PCIe. Not one pool of 384. This is what "local AI" looks like at the top end. A datacenter rack that happens to be in a room. So before the GPU budget — what does your panel actually deliver?

21,650 просмотров • 1 месяц назад

$15,000 for a Mac sounds insane. Until you realize what Apple is actually selling. 256GB of unified memory. Not a huge GPU farm. Not an NVIDIA rack. Not 4 different machines. Just one box that can keep multiple large LLMs in memory and run them locally. That changes the equation for local AI. NVIDIA wins when you need raw GPU compute. Apple gets interesting when the bottleneck is memory capacity. Because once your model doesn't fit into 24GB, 32GB or 48GB of VRAM, buying a faster GPU doesn't solve the problem. You need more memory. And that's where machines like the Mac Studio start looking less crazy. The interesting question isn't: “Is a $15K Mac faster than an NVIDIA GPU?” It isn't. The real question is: “How much are you willing to pay to make huge local models actually fit?” That’s where unified memory gets interesting.

$15,000 for a Mac sounds insane. Until you realize what Apple is actually selling. 256GB of unified memory. Not a huge GPU farm. Not an NVIDIA rack. Not 4 different machines. Just one box that can keep multiple large LLMs in memory and run them locally. That changes the equation for local AI. NVIDIA wins when you need raw GPU compute. Apple gets interesting when the bottleneck is memory capacity. Because once your model doesn't fit into 24GB, 32GB or 48GB of VRAM, buying a faster GPU doesn't solve the problem. You need more memory. And that's where machines like the Mac Studio start looking less crazy. The interesting question isn't: “Is a $15K Mac faster than an NVIDIA GPU?” It isn't. The real question is: “How much are you willing to pay to make huge local models actually fit?” That’s where unified memory gets interesting.

19,536 просмотров • 2 месяцев назад

Больше нет контента для загрузки