Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

BOLT A GPU ONTO THIS $200 FANLESS BOARD AND IT DOES THE TWO THINGS A RASPBERRY PI PHYSICALLY CAN'T: LOCAL AI AND 4K TRANSCODING. that clip is a zimaboard with a graphics card plugged straight into its pcie slot. that slot is the whole point. a pi has gpio... and usb, but no real pcie, so it tops out where this one starts. drop a gpu in and the tiny box suddenly does: ollama and local models with actual gpu acceleration, not cpu crawl plex or jellyfin transcoding several 4k streams at once, in hardware anything else that was "you need a real pc for that", on a fanless board the rest of the board backs it up: quad-core intel up to 3.6, dual sata, dual 2.5gbe, runs zimaos, linux, even windows. so one silent box becomes a nas, a media server, and a local ai node at the same time. no tower, no raspberry-pi ceiling, no cloud gpu rented by the hour. the uncomfortable part isn't that it takes a gpu. it's that "a pi can do it" quietly stopped being true the moment the workload needed a real pcie lane. follow RetroChainer for the small boxes that punch way above their size, and bookmark this before you buy a whole tower for one gpu job.show more

RetroChainer

6,217 subscribers

24,813 просмотров • 3 месяцев назад •via X (Twitter)

Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 3

Фото профиля AI Mastery Guide

AI Mastery Guide3 месяцев назад

$200 for that setup is a steal

Фото профиля Briggsll2262

Briggsll22623 месяцев назад

that pcie slot turns it from a tinker toy into something that actually earns its keep

Фото профиля gkrovso

gkrovso3 месяцев назад

wonder how it handles thermals with a gpu

Похожие видео

AMD JUST KILLED THE $200 - A - MONTH AI SUBSCRIPTION - A 65-WATT BOX WITH 128GB OF UNIFIED MEMORY THAT RUNS 70B MODELS LOCALLY. that clip is amd's ryzen ai max+ 395 mini box, held up on stage. the whole pitch is one spec: 128gb of high-speed unified memory. why it matters: cpu, gpu and npu share the same 128gb pool, so a model that used to need a rack of vram just fits. what it actually runs: llama 70b in full precision on the built-in gpu open models up to ~100b parameters, locally, no cloud all of it in a mini pc pulling around 65 watts that's the part that ends the subscription: your prompts, your data, your model, on a box on your desk, not metered by the token. honest line: this runs open models, not gpt-5 or claude opus. for chat, coding help and agents on your own data it's more than enough. for the absolute frontier, the cloud still wins. and it isn't free: the box runs ~$1,500 and up. but that's a few months of a $200 sub, then it's yours forever. no monthly ai bill, no tokens metered, no prompt leaving the room. the uncomfortable part isn't that amd made a fast box. it's that "you need the cloud for ai" was a pricing choice, and unified memory just made it optional. follow RetroChainer for the hardware that turns rent into ownership, and bookmark this before your next ai renewal.

AMD JUST KILLED THE $200 - A - MONTH AI SUBSCRIPTION - A 65-WATT BOX WITH 128GB OF UNIFIED MEMORY THAT RUNS 70B MODELS LOCALLY. that clip is amd's ryzen ai max+ 395 mini box, held up on stage. the whole pitch is one spec: 128gb of high-speed unified memory. why it matters: cpu, gpu and npu share the same 128gb pool, so a model that used to need a rack of vram just fits. what it actually runs: llama 70b in full precision on the built-in gpu open models up to ~100b parameters, locally, no cloud all of it in a mini pc pulling around 65 watts that's the part that ends the subscription: your prompts, your data, your model, on a box on your desk, not metered by the token. honest line: this runs open models, not gpt-5 or claude opus. for chat, coding help and agents on your own data it's more than enough. for the absolute frontier, the cloud still wins. and it isn't free: the box runs ~$1,500 and up. but that's a few months of a $200 sub, then it's yours forever. no monthly ai bill, no tokens metered, no prompt leaving the room. the uncomfortable part isn't that amd made a fast box. it's that "you need the cloud for ai" was a pricing choice, and unified memory just made it optional. follow RetroChainer for the hardware that turns rent into ownership, and bookmark this before your next ai renewal.

RetroChainer

16,995 просмотров • 3 месяцев назад

7-YEAR-OLD RASPBERRY PI WITH 2GB OF RAM IS RUNNING DEEPSEEK RIGHT NOW WHILE SAM ALTMAN BILLS $200/MO AND DARIO AMODEI BILLS $200/MO FOR A MODEL YOU NEVER GET TO KEEP 00:34 he points at the smallest board on the bench and says "out of all these I'm genuinely shocked that I actually got it up and running on this raspberry pi 4b with 2 gigs of ram" raspberry pi 4b, 2gb, seven years old -> ollama with deepseek r1 1.5b -> a dell micro form factor doing cpu-only inference next to it -> an m1 max macbook with 64gb unified memory beating the dual gpu tower on large models -> four machines, four architectures, all serving locally apple's unified memory puts the ram on the same die as the cpu and gpu, so a laptop outruns a two-card workstation on big models, and the cheapest board on that bench costs less than four days of a claude max seat the honest part: 2gb is the actual floor and he says it was rough to get there, a 4gb or 8gb pi would have taken minutes instead, and 1.5b parameters is a model you use for structure not for answers raspberry pi 4b 2gb + dell micro form factor + m1 max 64gb + dual gpu tower + ollama + deepseek r1 watch and save it, musk gave away the agent and moonshot gave away the weights, the full $400 to $25 routing table is in the article below

7-YEAR-OLD RASPBERRY PI WITH 2GB OF RAM IS RUNNING DEEPSEEK RIGHT NOW WHILE SAM ALTMAN BILLS $200/MO AND DARIO AMODEI BILLS $200/MO FOR A MODEL YOU NEVER GET TO KEEP 00:34 he points at the smallest board on the bench and says "out of all these I'm genuinely shocked that I actually got it up and running on this raspberry pi 4b with 2 gigs of ram" raspberry pi 4b, 2gb, seven years old -> ollama with deepseek r1 1.5b -> a dell micro form factor doing cpu-only inference next to it -> an m1 max macbook with 64gb unified memory beating the dual gpu tower on large models -> four machines, four architectures, all serving locally apple's unified memory puts the ram on the same die as the cpu and gpu, so a laptop outruns a two-card workstation on big models, and the cheapest board on that bench costs less than four days of a claude max seat the honest part: 2gb is the actual floor and he says it was rough to get there, a 4gb or 8gb pi would have taken minutes instead, and 1.5b parameters is a model you use for structure not for answers raspberry pi 4b 2gb + dell micro form factor + m1 max 64gb + dual gpu tower + ollama + deepseek r1 watch and save it, musk gave away the agent and moonshot gave away the weights, the full $400 to $25 routing table is in the article below

Antid

10,442 просмотров • 2 месяцев назад

THIS $1,300 CARD HAS THE SAME 32GB OF VRAM AS A $2,000+ RTX 5090 - FOR LOCAL AI, THAT'S THE SPEC THAT DECIDES EVERYTHING. that clip is the gigabyte radeon ai pro r9700 ai top, 32gb of vram, built for local ai. the pitch is simple: the same memory capacity as an rtx 5090, for roughly a third of what a 5090 actually sells for. here's why that matters, and where it doesn't. for ai, vram is the gate. a model either fits in memory or it doesn't. 32gb loads models a 16gb card can't touch, full stop. so for running local llms and image models, this card does the one thing that counts, cheap. honest line: this is not a 5090 for gaming. the 5090 is far faster at raster and ray tracing. they only match on how much vram they carry. if you want frames, buy the 5090. if you want to fit big models locally, you just found the same 32gb for a third of the money. no $2,000+ price tag, no cloud gpu rental, no model too big for 16gb. the uncomfortable part isn't that amd is cheaper. it's that for local ai you were about to overpay for gaming power you'd never use. follow RetroChainer for more on paying for the spec that matters, and bookmark this before your next gpu.

THIS $1,300 CARD HAS THE SAME 32GB OF VRAM AS A $2,000+ RTX 5090 - FOR LOCAL AI, THAT'S THE SPEC THAT DECIDES EVERYTHING. that clip is the gigabyte radeon ai pro r9700 ai top, 32gb of vram, built for local ai. the pitch is simple: the same memory capacity as an rtx 5090, for roughly a third of what a 5090 actually sells for. here's why that matters, and where it doesn't. for ai, vram is the gate. a model either fits in memory or it doesn't. 32gb loads models a 16gb card can't touch, full stop. so for running local llms and image models, this card does the one thing that counts, cheap. honest line: this is not a 5090 for gaming. the 5090 is far faster at raster and ray tracing. they only match on how much vram they carry. if you want frames, buy the 5090. if you want to fit big models locally, you just found the same 32gb for a third of the money. no $2,000+ price tag, no cloud gpu rental, no model too big for 16gb. the uncomfortable part isn't that amd is cheaper. it's that for local ai you were about to overpay for gaming power you'd never use. follow RetroChainer for more on paying for the spec that matters, and bookmark this before your next gpu.

RetroChainer

38,844 просмотров • 3 месяцев назад

THE MS-A2 IS THE MINISFORUM BOX HERMES AGENT WAS NEVER MEANT TO RUN ON, AND THAT'S BY DESIGN The MS-S1 MAX earns its spot running Hermes Agent because of one thing. 128GB of unified memory, up to 96GB of it handed straight to the GPU. That's the only reason a 120B model fits and runs locally for $0 a month The MS-A2 solves a different problem. Ryzen 9 9955HX, 16 cores, 32 threads, up to 96GB of regular DDR5-5600, no unified pool Three M.2 PCIe 4.0 slots, one U.2, two 22110. Dual 10Gbps SFP+ LAN plus 2.5G. WiFi 6E. Bluetooth 5.3. A slide-out motherboard for fast upgrades. A real PCIe x16 slot that actually takes a low-profile GPU That last part is where the two machines split for good. The MS-S1 MAX's PCIe slot won't take a GPU at all, every bit of GPU power has to come from the unified chip itself. The MS-A2 trades that unified memory trick for raw expandability instead One box runs a local AI agent. The other runs a home lab that needs storage, networking, and room to grow. Minisforum built both on purpose, not as the same product wearing two names

NO1ennn

25,645 просмотров • 4 месяцев назад

EVERYONE'S BUYING THE M4 MAC MINI AS A CHEAP DESKTOP. HERE'S WHY IT ENDS UP UNUSED. the pitch is irresistible: a tiny mac that runs cool, quiet, and cheap. then the receipt shows up. the mac mini ships with no screen, no keyboard, no mouse. add a monitor, a decent keyboard and mouse, and more storage, and the "$799 mac" is past $1,000 fast. and macos isn't windows. if your world is gaming, specific windows apps, or the setup you already know, the mini fights you the whole way. that's how so many end up back on a pc, the mini in a drawer. so what is the mac mini actually great at? not being your main desktop. it's the box you never look at. headless in a corner, it's a near-silent always-on server: run local ai models on it, host your automations, keep your files, and reach it from the laptop you already use. it earns its keep by disappearing. the honest turn: the mac mini isn't a cheap desktop. it's a cheap server pretending to be one. buy it for the job it's quietly brilliant at, not the one the ads imply. want a main computer? price the whole setup first. want a silent little box that runs ai and never bothers you? this is the one. no monitor in the box, no keyboard, no "just $599". save this before you buy the sticker price instead of the real one.

EVERYONE'S BUYING THE M4 MAC MINI AS A CHEAP DESKTOP. HERE'S WHY IT ENDS UP UNUSED. the pitch is irresistible: a tiny mac that runs cool, quiet, and cheap. then the receipt shows up. the mac mini ships with no screen, no keyboard, no mouse. add a monitor, a decent keyboard and mouse, and more storage, and the "$799 mac" is past $1,000 fast. and macos isn't windows. if your world is gaming, specific windows apps, or the setup you already know, the mini fights you the whole way. that's how so many end up back on a pc, the mini in a drawer. so what is the mac mini actually great at? not being your main desktop. it's the box you never look at. headless in a corner, it's a near-silent always-on server: run local ai models on it, host your automations, keep your files, and reach it from the laptop you already use. it earns its keep by disappearing. the honest turn: the mac mini isn't a cheap desktop. it's a cheap server pretending to be one. buy it for the job it's quietly brilliant at, not the one the ads imply. want a main computer? price the whole setup first. want a silent little box that runs ai and never bothers you? this is the one. no monitor in the box, no keyboard, no "just $599". save this before you buy the sticker price instead of the real one.

RetroChainer

16,506 просмотров • 3 месяцев назад

$3,800 for a Mini PC with 96GB of GPU memory sounds ridiculous. Until you see what AMD is actually doing. The GMKtec EVO-X3 has: Ryzen AI MAX+ 395 128GB LPDDR5X Up to 96GB allocated to the Radeon 8060S 8000 MT/s memory No discrete GPU. That means a tiny box can run local models that would normally push you toward a workstation with a huge GPU. And it can still play games. The interesting part isn't the 126 TOPS number. It's the fact that 96GB of GPU memory now fits inside a mini PC. The question is no longer whether this can run a big model. It's how fast it can run it. The EVO-X3 is available here:

$3,800 for a Mini PC with 96GB of GPU memory sounds ridiculous. Until you see what AMD is actually doing. The GMKtec EVO-X3 has: Ryzen AI MAX+ 395 128GB LPDDR5X Up to 96GB allocated to the Radeon 8060S 8000 MT/s memory No discrete GPU. That means a tiny box can run local models that would normally push you toward a workstation with a huge GPU. And it can still play games. The interesting part isn't the 126 TOPS number. It's the fact that 96GB of GPU memory now fits inside a mini PC. The question is no longer whether this can run a big model. It's how fast it can run it. The EVO-X3 is available here:

KyzoroX

24,654 просмотров • 1 месяц назад

ANTHROPIC CHARGES $200 A MONTH SO SOMETHING IS ALWAYS AWAKE FOR YOU. THIS FANLESS BOARD DOES THAT AT ROUGHLY 8 WATTS AND WAKES THE EXPENSIVE MACHINE ONLY WHEN A REQUEST ACTUALLY ARRIVES here is the whole shape, always-on to burst: a Zima Board 2, passively cooled in a machined case, silent by design -> dual 2.5 gigabit ethernet, two SATA ports, a PCIe Gen3 slot on the side -> it sits on 24/7 and costs almost nothing to leave running -> it holds the queue, the router and the API endpoint -> when real work lands it wakes the big rig, hands off, and the big rig sleeps again. the expensive part of local AI is not the tokens, it is the idle. a workstation doing nothing still pulls 40 to 120 watts, and that is the number that turned a $1,920 pallet into a $6,000 power bill in the last video. the always-on box and the compute box should not be the same machine. that is the whole idea, and it is five things: separate the front door from the factory -> a fanless 8W board can hold your endpoint, your queue and your web UI forever -> the GPU machine only needs to exist during the seconds it is actually generating idle watts compound, peak watts do not -> 40W of idle for a year is roughly the same money as a few hundred hours of real inference -> nobody measures idle because it never shows up in a benchmark dual 2.5GbE is the feature that makes this work -> one port to your network, one to the compute box, so wake-on-lan and handoff are clean -> this is the difference between a hobby setup and something the rest of the house can actually use the PCIe slot means it can do small jobs itself -> a low power card in that slot handles embeddings, classification and speech to text without waking anything -> route by size: tiny jobs die here, big jobs get escalated passive cooling is a reliability spec -> no fan means no fan failure, which matters for the one machine that is never supposed to go down -> the loud rig can be loud, it is only awake for minutes a day openai and anthropic sell you a warm machine that is always ready. that readiness is real, and it costs them a fortune at scale. at home it costs about 8 watts. my take, and it is the uncomfortable one: most people who priced local AI compared a sleeping subscription to a rig that never sleeps, and the fix is two machines instead of one. read the article below. that's how you stop paying for idle.

ANTHROPIC CHARGES $200 A MONTH SO SOMETHING IS ALWAYS AWAKE FOR YOU. THIS FANLESS BOARD DOES THAT AT ROUGHLY 8 WATTS AND WAKES THE EXPENSIVE MACHINE ONLY WHEN A REQUEST ACTUALLY ARRIVES here is the whole shape, always-on to burst: a Zima Board 2, passively cooled in a machined case, silent by design -> dual 2.5 gigabit ethernet, two SATA ports, a PCIe Gen3 slot on the side -> it sits on 24/7 and costs almost nothing to leave running -> it holds the queue, the router and the API endpoint -> when real work lands it wakes the big rig, hands off, and the big rig sleeps again. the expensive part of local AI is not the tokens, it is the idle. a workstation doing nothing still pulls 40 to 120 watts, and that is the number that turned a $1,920 pallet into a $6,000 power bill in the last video. the always-on box and the compute box should not be the same machine. that is the whole idea, and it is five things: separate the front door from the factory -> a fanless 8W board can hold your endpoint, your queue and your web UI forever -> the GPU machine only needs to exist during the seconds it is actually generating idle watts compound, peak watts do not -> 40W of idle for a year is roughly the same money as a few hundred hours of real inference -> nobody measures idle because it never shows up in a benchmark dual 2.5GbE is the feature that makes this work -> one port to your network, one to the compute box, so wake-on-lan and handoff are clean -> this is the difference between a hobby setup and something the rest of the house can actually use the PCIe slot means it can do small jobs itself -> a low power card in that slot handles embeddings, classification and speech to text without waking anything -> route by size: tiny jobs die here, big jobs get escalated passive cooling is a reliability spec -> no fan means no fan failure, which matters for the one machine that is never supposed to go down -> the loud rig can be loud, it is only awake for minutes a day openai and anthropic sell you a warm machine that is always ready. that readiness is real, and it costs them a fortune at scale. at home it costs about 8 watts. my take, and it is the uncomfortable one: most people who priced local AI compared a sleeping subscription to a rig that never sleeps, and the fix is two machines instead of one. read the article below. that's how you stop paying for idle.

Antid

13,140 просмотров • 1 месяц назад

YOU DON'T NEED A SERVER RACK. SIX USED MINI PCS AND A $60 CISCO SWITCH WIRE INTO A HOME CLUSTER ON YOUR DESK. that clip is six tiny mini-pcs stacked and patched into one cisco business 110 switch, red cables, a single power strip. each box is a full x86 computer, the kind offices dump by the pallet. used, they run $30 to $60 each. six of them plus the switch is a real cluster for a couple hundred dollars. what a stack like this runs: proxmox or kubernetes spread across all six nodes pi-hole, a nas, home assistant, docker services, each on its own node a local model or two on the boxes with enough ram the trick isn't raw power, it's separation. one node dies, the cluster keeps going. one service crashes, it doesn't drag the rest down. no server rack, no datacenter noise, no monthly cloud bill. the uncomfortable part isn't that it's cheap. it's that the "real infrastructure" you were told needs a rack and a budget fits on a desk and boots off a power strip. follow RetroChainer for more ways to build real infrastructure for pocket money, and bookmark this before the next office dumps its mini pcs.

YOU DON'T NEED A SERVER RACK. SIX USED MINI PCS AND A $60 CISCO SWITCH WIRE INTO A HOME CLUSTER ON YOUR DESK. that clip is six tiny mini-pcs stacked and patched into one cisco business 110 switch, red cables, a single power strip. each box is a full x86 computer, the kind offices dump by the pallet. used, they run $30 to $60 each. six of them plus the switch is a real cluster for a couple hundred dollars. what a stack like this runs: proxmox or kubernetes spread across all six nodes pi-hole, a nas, home assistant, docker services, each on its own node a local model or two on the boxes with enough ram the trick isn't raw power, it's separation. one node dies, the cluster keeps going. one service crashes, it doesn't drag the rest down. no server rack, no datacenter noise, no monthly cloud bill. the uncomfortable part isn't that it's cheap. it's that the "real infrastructure" you were told needs a rack and a budget fits on a desk and boots off a power strip. follow RetroChainer for more ways to build real infrastructure for pocket money, and bookmark this before the next office dumps its mini pcs.

RetroChainer

95,267 просмотров • 3 месяцев назад

THIS WHOLE 3-NODE CLUSTER RUNS PROXMOX, CEPH AND KUBERNETES - ON 10 WATTS TOTAL. that clip is three zima boards wired to one little switch, and the power meter reads 10 watts for the entire thing. on those three tiny fanless boards: proxmox running a real cluster, three nodes as one datacenter ceph spreading storage across all three, so a node can die and your data doesn't kubernetes on top, kube1/2/3, plus docker and high-availability failover this is the exact stack that runs in datacenters, shrunk onto a shelf. kill one node and the vms migrate to the other two. it heals itself. and it pulls less power than a light bulb. no server rack, no cooling, no power bill you'd ever notice. the uncomfortable part isn't that it's small. it's that "enterprise infrastructure" was mostly the rack and the noise, and none of that was the actual capability. follow RetroChainer for more ways enterprise gear shrinks to a shelf, and bookmark this before you rent another vps.

THIS WHOLE 3-NODE CLUSTER RUNS PROXMOX, CEPH AND KUBERNETES - ON 10 WATTS TOTAL. that clip is three zima boards wired to one little switch, and the power meter reads 10 watts for the entire thing. on those three tiny fanless boards: proxmox running a real cluster, three nodes as one datacenter ceph spreading storage across all three, so a node can die and your data doesn't kubernetes on top, kube1/2/3, plus docker and high-availability failover this is the exact stack that runs in datacenters, shrunk onto a shelf. kill one node and the vms migrate to the other two. it heals itself. and it pulls less power than a light bulb. no server rack, no cooling, no power bill you'd ever notice. the uncomfortable part isn't that it's small. it's that "enterprise infrastructure" was mostly the rack and the noise, and none of that was the actual capability. follow RetroChainer for more ways enterprise gear shrinks to a shelf, and bookmark this before you rent another vps.

RetroChainer

42,348 просмотров • 3 месяцев назад

A USED $700 RTX 3090 HAS MORE MEMORY AND FAR MORE SPEED THAN THE ~$1,000 A2 GOING INTO THIS SERVER. THE A2 STILL WINS - FOR ONE REASON THAT ISN'T PERFORMANCE. that clip is a dell poweredge r760 - a 2u rack server - getting an nvidia a2 dropped in through a riser. this is the enterprise route to local ai. the rung the desktop ladder skips entirely. the a2, verified: 16gb of memory, single-slot, low-profile, and just 40 to 60 watts it's an inference card, built to sit in a server that has no room or spare power for a real gpu so why not a 3090? a used rtx 3090 gives you 24gb and many times the compute for around $700. the a2 gives 16gb, much slower, for roughly a thousand. on raw local-llm value, the desktop card wins, and it isn't close. the a2's whole reason to exist is the thing you can't see on a spec sheet: it fits. a 2u server has no spare gpu power and no room for a triple-slot furnace. the a2 slips into one low-profile slot on 60 watts and lets an existing server do inference without a rebuild. the honest caveats: the r760 is not a desk machine. loaded, it idles at hundreds of watts, and its fans sound like a hair dryer that never turns off. it belongs in a rack or a closet 16gb is still 16gb. the same 7b-to-32b ceiling as the cheap cards. the enterprise badge doesn't buy you a bigger model what it does buy: ecc memory, redundant power, remote management, hot-swap everything. reliability, not speed so who this is for: someone who already runs a rack and wants to add private inference without touching the power or cooling budget. for anyone starting from a desk, the mac mini or the 3090 wins on every axis that matters. the real point: the "best" local ai box depends entirely on what you already own. on a desk, efficiency wins. in a rack, fit and reliability win. the a2 isn't a bad card - it's a card for a constraint the ladder never mentions. no fastest gpu here, no desk-friendly box, no bigger model than the cheap cards already run. save this before you buy enterprise gear for a desktop job.

Grimmer

24,287 просмотров • 3 месяцев назад

STARLINK CHARGES $120/MO TO GET ONLINE OFF-GRID. HE DOES IT FOR $0 - A $30 PANEL ANTENNA ON A MAST REACHING A USED GPU 80KM OUT. RUNS OPUS-5 AND GPT-5.6 LOCALLY he clamped a directional panel and a tp-link adapter to a telescoping mast and ran it up past the roofline, whole rig cost him $30 in used marketplace parts the higher it goes the farther it hears -> it locks a node 80km out -> the prompt tunnels down to the used gpu in his closet -> a local kimi k3 answers before the page finishes loading one node it kept pulling in stayed online through last month's grid blackout, turned out to be a repeater on a water tower a county away running off a small solar pack height is the whole cheat, lift a directional antenna over the treeline and line-of-sight does the rest, every node it reaches is just another road home to his own gpu $200 opus-5 and $200 gpt-5.6 pro stopped hitting his card the week that mast went up, a $30 antenna and a used gpu do what anthropic and openai charged $400 a month for full breakdown of the 5-device stack that zeroes your ai bill in the article below

STARLINK CHARGES $120/MO TO GET ONLINE OFF-GRID. HE DOES IT FOR $0 - A $30 PANEL ANTENNA ON A MAST REACHING A USED GPU 80KM OUT. RUNS OPUS-5 AND GPT-5.6 LOCALLY he clamped a directional panel and a tp-link adapter to a telescoping mast and ran it up past the roofline, whole rig cost him $30 in used marketplace parts the higher it goes the farther it hears -> it locks a node 80km out -> the prompt tunnels down to the used gpu in his closet -> a local kimi k3 answers before the page finishes loading one node it kept pulling in stayed online through last month's grid blackout, turned out to be a repeater on a water tower a county away running off a small solar pack height is the whole cheat, lift a directional antenna over the treeline and line-of-sight does the rest, every node it reaches is just another road home to his own gpu $200 opus-5 and $200 gpt-5.6 pro stopped hitting his card the week that mast went up, a $30 antenna and a used gpu do what anthropic and openai charged $400 a month for full breakdown of the 5-device stack that zeroes your ai bill in the article below

starmex

144,506 просмотров • 3 месяцев назад

THIS CHINESE BUILDER TURNED 8 RETIRED TESLA P40s INTO A 192GB PRIVATE AI SERVER THAT CAN WORK 5,760 GPU-HOURS EVERY MONTH. 00:06 he holds up one used Tesla P40 while seven more sit behind it, then installs the full stack inside a massive dual-Xeon server chassis. each card carries 24GB of VRAM, giving the system 192GB across eight GPUs for local models, video processing, transcription, private documents and AI agents. the real advantage is parallel work. some cards can serve a local model while the rest generate images, process files and run overnight automation at the same time. eight GPUs running continuously deliver 5,760 GPU-hours every 30 days. rent that workload from the cloud and the meter keeps moving for every minute the server stays online. a $412 monthly AI stack already costs $4,944 a year before serious GPU rental. he bought retired hardware once, kept the data local and built a machine that keeps working after the browser closes.

THIS CHINESE BUILDER TURNED 8 RETIRED TESLA P40s INTO A 192GB PRIVATE AI SERVER THAT CAN WORK 5,760 GPU-HOURS EVERY MONTH. 00:06 he holds up one used Tesla P40 while seven more sit behind it, then installs the full stack inside a massive dual-Xeon server chassis. each card carries 24GB of VRAM, giving the system 192GB across eight GPUs for local models, video processing, transcription, private documents and AI agents. the real advantage is parallel work. some cards can serve a local model while the rest generate images, process files and run overnight automation at the same time. eight GPUs running continuously deliver 5,760 GPU-hours every 30 days. rent that workload from the cloud and the meter keeps moving for every minute the server stays online. a $412 monthly AI stack already costs $4,944 a year before serious GPU rental. he bought retired hardware once, kept the data local and built a machine that keeps working after the browser closes.

Gipp 🦅

42,332 просмотров • 4 месяцев назад

THIS GUY WIRED GMKTEC EVO-X2 BOXES, A $40 USB-C BRICK AND AN RTX 3090 INTO A $6,800/MONTH OFFLINE AI FARM it looks like a messy rack with purple cables, mini pcs and a gpu lying on the desk. but the real point is brutal: one $1,800 gmktec evo-x2 can replace a $440/month ai stack, then the rack turns it into a local backend. the evo-x2 runs amd’s ryzen ai max+ 395 with 128gb of unified memory shared between cpu, gpu and npu. on linux, that gives up to 110gb of usable vram for local models without renting cloud compute. that is why the setup gets weird. qwen3-coder 30b, llama 3.3 70b, deepseek v3 and 200b-class models can run from boxes smaller than consoles. the rtx 3090 becomes a 24gb side node, not the whole system. the $40 anker nano ii 65w brick is the funny part. cheap power, tiny machines, one old gpu and local inference. the whole stack starts looking less like a workstation and more like a private ai grid. the subscription math is ugly. claude code max is $200/month, chatgpt pro is $200/month, cursor is $20/month and gemini adds another $20. that is $5,280/year before api usage even starts. point claude code at localhost, pull models through ollama and run agents all day: code reviews, scraping, transcription, research reports and client automations. 3 clients at $2,200/month turns this into a $6,800/month backend. bookmark this and read the article below

THIS GUY WIRED GMKTEC EVO-X2 BOXES, A $40 USB-C BRICK AND AN RTX 3090 INTO A $6,800/MONTH OFFLINE AI FARM it looks like a messy rack with purple cables, mini pcs and a gpu lying on the desk. but the real point is brutal: one $1,800 gmktec evo-x2 can replace a $440/month ai stack, then the rack turns it into a local backend. the evo-x2 runs amd’s ryzen ai max+ 395 with 128gb of unified memory shared between cpu, gpu and npu. on linux, that gives up to 110gb of usable vram for local models without renting cloud compute. that is why the setup gets weird. qwen3-coder 30b, llama 3.3 70b, deepseek v3 and 200b-class models can run from boxes smaller than consoles. the rtx 3090 becomes a 24gb side node, not the whole system. the $40 anker nano ii 65w brick is the funny part. cheap power, tiny machines, one old gpu and local inference. the whole stack starts looking less like a workstation and more like a private ai grid. the subscription math is ugly. claude code max is $200/month, chatgpt pro is $200/month, cursor is $20/month and gemini adds another $20. that is $5,280/year before api usage even starts. point claude code at localhost, pull models through ollama and run agents all day: code reviews, scraping, transcription, research reports and client automations. 3 clients at $2,200/month turns this into a $6,800/month backend. bookmark this and read the article below

Gipp 🦅

20,031 просмотров • 4 месяцев назад

YOUR HOMELAB IS USELESS IF YOU CAN'T REACH IT FROM ANYWHERE - AND YOU SHOULD NEVER OPEN A PORT TO DO IT. that clip is a homelab in a tiny rack: cheap used mini-pcs running a proxmox cluster with ollama for local ai. but the hardware isn't the hard part. reaching it safely when you're not home is. the wrong way: forward a port on your router. now your services face the whole internet, and every bot on earth starts knocking. the right way, and what he's running: a mesh vpn like twingate, netbird or tailscale. it builds a private encrypted network between your devices and the lab. no open ports, no public ip, nothing exposed. your phone in a cafe talks to your proxmox box at home like they're on the same wifi, and the internet sees nothing. that's the piece that turns a pile of mini-pcs into infrastructure you actually use. no port forwarding, no exposed dashboard, no public attack surface. the uncomfortable part isn't building the lab. it's that most people either can't reach it, or reach it by leaving the front door wide open. follow RetroChainer for the homelab step everyone skips, and bookmark this before you forward a single port.

YOUR HOMELAB IS USELESS IF YOU CAN'T REACH IT FROM ANYWHERE - AND YOU SHOULD NEVER OPEN A PORT TO DO IT. that clip is a homelab in a tiny rack: cheap used mini-pcs running a proxmox cluster with ollama for local ai. but the hardware isn't the hard part. reaching it safely when you're not home is. the wrong way: forward a port on your router. now your services face the whole internet, and every bot on earth starts knocking. the right way, and what he's running: a mesh vpn like twingate, netbird or tailscale. it builds a private encrypted network between your devices and the lab. no open ports, no public ip, nothing exposed. your phone in a cafe talks to your proxmox box at home like they're on the same wifi, and the internet sees nothing. that's the piece that turns a pile of mini-pcs into infrastructure you actually use. no port forwarding, no exposed dashboard, no public attack surface. the uncomfortable part isn't building the lab. it's that most people either can't reach it, or reach it by leaving the front door wide open. follow RetroChainer for the homelab step everyone skips, and bookmark this before you forward a single port.

RetroChainer

61,925 просмотров • 3 месяцев назад

THEY RAN A LOCAL AI MODEL ON A RASPBERRY PI THAT COSTS LESS THAN $300 no cloud no subscription no data center device smaller than your wallet output on a 27 year old iMac I tested local ai this week on a macbook and a phone the hardware keeps getting cheaper and the models keep getting smarter bookmark this if you’re still paying $20 a month for something your $300 raspberry pi can do

THEY RAN A LOCAL AI MODEL ON A RASPBERRY PI THAT COSTS LESS THAN $300 no cloud no subscription no data center device smaller than your wallet output on a 27 year old iMac I tested local ai this week on a macbook and a phone the hardware keeps getting cheaper and the models keep getting smarter bookmark this if you’re still paying $20 a month for something your $300 raspberry pi can do

leopardracer

73,534 просмотров • 5 месяцев назад

100 MAC MINIS ON ONE METAL SHELF. IN THE CLOUD, EACH OF THOSE RENTS FOR $120 TO $500 A MONTH. that clip is a homemade rack: wire shelving packed with mac minis, each running macos, wired into one cluster. why macs and not a normal server? two reasons. you can only build, sign, and notarize ios and mac apps on apple hardware. cloud providers know it and charge for it. and apple silicon sips power and runs near-silent, so you can stack a lot of them on a shelf in a room, not a datacenter. the rent they're escaping: a single mac in the cloud runs ~$120 to $500 a month (macstadium up to aws) one engineer documented saving $4,000+ a month by self-hosting mac minis instead of cloud ci runners a used or new mini is a few hundred dollars once, then a few dollars of electricity what a shelf like this actually runs: ios and mac build farms, render jobs, app and device testing automation, and increasingly local ai inference. own the boxes, rent nothing. the uncomfortable part: the cloud was never the only option. it was the convenient one. the people who did the math bought the shelf. the honest caveat: it's real capex up front, real heat and power, and you become the sysadmin. no rented rack means no one to call at 3am. worth it at scale, overkill for one build a week. no rack rental, no per-hour metering, no fleet you don't own. save this before your cloud invoice renews again.

100 MAC MINIS ON ONE METAL SHELF. IN THE CLOUD, EACH OF THOSE RENTS FOR $120 TO $500 A MONTH. that clip is a homemade rack: wire shelving packed with mac minis, each running macos, wired into one cluster. why macs and not a normal server? two reasons. you can only build, sign, and notarize ios and mac apps on apple hardware. cloud providers know it and charge for it. and apple silicon sips power and runs near-silent, so you can stack a lot of them on a shelf in a room, not a datacenter. the rent they're escaping: a single mac in the cloud runs ~$120 to $500 a month (macstadium up to aws) one engineer documented saving $4,000+ a month by self-hosting mac minis instead of cloud ci runners a used or new mini is a few hundred dollars once, then a few dollars of electricity what a shelf like this actually runs: ios and mac build farms, render jobs, app and device testing automation, and increasingly local ai inference. own the boxes, rent nothing. the uncomfortable part: the cloud was never the only option. it was the convenient one. the people who did the math bought the shelf. the honest caveat: it's real capex up front, real heat and power, and you become the sysadmin. no rented rack means no one to call at 3am. worth it at scale, overkill for one build a week. no rack rental, no per-hour metering, no fleet you don't own. save this before your cloud invoice renews again.

RetroChainer

21,055 просмотров • 3 месяцев назад

How many parameters does a voice need? Pi(co) Vo(ice): two English voices in 1.4M parameters, vocoder included. The whole model is 5.8 MB and runs on a CPU, no PyTorch or GPU.

How many parameters does a voice need? Pi(co) Vo(ice): two English voices in 1.4M parameters, vocoder included. The whole model is 5.8 MB and runs on a CPU, no PyTorch or GPU.

Empero

131,912 просмотров • 12 дней назад

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

Akshay 🚀

346,057 просмотров • 2 месяцев назад

$THIS IS WHERE "THE CLOUD" ACTUALLY LIVES - A ROOM OF DELL STORAGE ARRAYS YOU RENT A SLIVER OF EVERY MONTH. that clip is a real datacenter aisle, rack after rack of dell powerstore arrays, every hexagon panel packed with drives. this is "the cloud." not a metaphor, this room, plus the power, cooling and staff keeping it alive. when you pay for cloud storage, you rent a few gigabytes of these arrays. forever. the bill grows with every photo. here's the part they'd rather you not connect. these enterprise drives run on a schedule. data centers swap them out long before they fail, by the pallet. those decommissioned drives get tested and resold as recertified, the same 20tb enterprise disk for a fraction of new. so the exact hardware humming in this room ends up in a nas on your desk, minus the monthly rent. no cloud subscription, no data on someone else's array, no bill that grows with your library. the uncomfortable part isn't that the cloud is huge. it's that "the cloud" was always just this room, and you can buy a piece of it once instead of renting it forever. follow RetroChainer for what the cloud doesn't want you to picture, and bookmark this before your next storage renewal.$

THIS IS WHERE "THE CLOUD" ACTUALLY LIVES - A ROOM OF DELL STORAGE ARRAYS YOU RENT A SLIVER OF EVERY MONTH. that clip is a real datacenter aisle, rack after rack of dell powerstore arrays, every hexagon panel packed with drives. this is "the cloud." not a metaphor, this room, plus the power, cooling and staff keeping it alive. when you pay for cloud storage, you rent a few gigabytes of these arrays. forever. the bill grows with every photo. here's the part they'd rather you not connect. these enterprise drives run on a schedule. data centers swap them out long before they fail, by the pallet. those decommissioned drives get tested and resold as recertified, the same 20tb enterprise disk for a fraction of new. so the exact hardware humming in this room ends up in a nas on your desk, minus the monthly rent. no cloud subscription, no data on someone else's array, no bill that grows with your library. the uncomfortable part isn't that the cloud is huge. it's that "the cloud" was always just this room, and you can buy a piece of it once instead of renting it forever. follow RetroChainer for what the cloud doesn't want you to picture, and bookmark this before your next storage renewal.

RetroChainer

34,270 просмотров • 3 месяцев назад

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

plutos

1,240,308 просмотров • 5 месяцев назад