Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

🚨 I DON'T KNOW WHAT PEOPLE ARE WAITING FOR! This guy just deployed a full GPU compute farm in a data center Not renting cloud instances, not paying per-token, just racks of consumer GPUs on warehouse shelving doing AI inference 24/7 The setup is wild: open-frame rigs with rows...

0xCristal's profile picture

cristal

8,507 subscribers

16,305 görüntüleme • 3 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Successfully deployed Deepseek R1 Distilled 70B (AWQ) across 8x NVIDIA RTX 3080 10G GPUs, achieving 60 tokens/s with full tensor parallelism via PCIe. Total hardware cost: $6,400 This demonstrates that consumer GPUs can deliver substantial ML inference capabilities at a fraction of the cost of datacenter hardware. For perspective, a single A100 80G costs $17,550, while a H100 80G runs $25,000. Our testing validates an often-overlooked opportunity: millions of idle crypto mining rigs could be repurposed into a powerful, distributed AI infrastructure. The performance-to-cost ratio of consumer GPUs, especially when properly optimized for tensor operations, presents a compelling case for decentralized AI compute. We're excited to share more insights as we continue pushing the boundaries of consumer hardware in AI workloads.
1:20

Successfully deployed Deepseek R1 Distilled 70B (AWQ) across 8x NVIDIA RTX 3080 10G GPUs, achieving 60 tokens/s with full tensor parallelism via PCIe. Total hardware cost: $6,400 This demonstrates that consumer GPUs can deliver substantial ML inference capabilities at a fraction of the cost of datacenter hardware. For perspective, a single A100 80G costs $17,550, while a H100 80G runs $25,000. Our testing validates an often-overlooked opportunity: millions of idle crypto mining rigs could be repurposed into a powerful, distributed AI infrastructure. The performance-to-cost ratio of consumer GPUs, especially when properly optimized for tensor operations, presents a compelling case for decentralized AI compute. We're excited to share more insights as we continue pushing the boundaries of consumer hardware in AI workloads.

TensorBlock

16,219 görüntüleme • 1 yıl önce

WAREHOUSES THE SIZE OF A WALMART WITH 4,000 GPUS WERE EARNING $320K A MONTH MINING ETH, THE SAME RACKS NOW EARN $4M A MONTH RENTING TO AI STARTUPS ON VAST AI the warehouse in the clip is one of hundreds across the US, the operator at 6'6" tall for scale, electricity contracts and cooling already in place ethereum's merge in september 2022 killed gpu mining for everyone without a power plant, by 2025 alt coin mining was barely covering the electricity bill the same racks now host RTX 3090s and 4090s on vast ai, indie AI developers priced out of AWS rent them by the hour for inference and image generation a 4090 nets $500 to $1,000 a month renting to AI startups, the same card was clearing $80 to $150 mining, 4 to 7 times the revenue with zero new hardware core scientific, hut 8 and riot have all signed multi billion dollar AI hosting deals in 2025, the warehouses you scrolled past on tiktok are quietly becoming the inference backbone of every AI tool you use bookmark this and read the article below
1:19

WAREHOUSES THE SIZE OF A WALMART WITH 4,000 GPUS WERE EARNING $320K A MONTH MINING ETH, THE SAME RACKS NOW EARN $4M A MONTH RENTING TO AI STARTUPS ON VAST AI the warehouse in the clip is one of hundreds across the US, the operator at 6'6" tall for scale, electricity contracts and cooling already in place ethereum's merge in september 2022 killed gpu mining for everyone without a power plant, by 2025 alt coin mining was barely covering the electricity bill the same racks now host RTX 3090s and 4090s on vast ai, indie AI developers priced out of AWS rent them by the hour for inference and image generation a 4090 nets $500 to $1,000 a month renting to AI startups, the same card was clearing $80 to $150 mining, 4 to 7 times the revenue with zero new hardware core scientific, hut 8 and riot have all signed multi billion dollar AI hosting deals in 2025, the warehouses you scrolled past on tiktok are quietly becoming the inference backbone of every AI tool you use bookmark this and read the article below

starmex

428,022 görüntüleme • 3 ay önce

ONE OPERATOR STACKED 300 GPUS ACROSS TWO APARTMENTS IN THE SAME BUILDING AND RUNS A $48K/MONTH AI INFERENCE FARM ON VAST AI FROM HIS LIVING ROOM 00:17 he walks past stacks of GPU boxes, "and probably another 100 GPU boxes in the second apartment, let me know in the comments if you want to see them" he rents 2 units in the same building, one as his living space with 200 GPUs in the bedroom and hallway, the second is dedicated and climate controlled just for the other 100 cards a 300 RTX 4090 setup pulls 135 kilowatts fully loaded, his power bill runs $9,800 a month at $0.10 per kwh, on vast ai the same fleet clears $48,000 in gross monthly rental income he never built this in a warehouse because residential electricity in his city is cheaper than commercial under 150 kw, the split apartment trick keeps him under that ceiling while doubling his rack space the same hardware would have cleared maybe $9,000 a month mining ethereum classic in 2022, vast ai pays 5 times that for AI inference because nobody can ship enough H100s to meet startup demand bookmark this and read the article below
0:20

ONE OPERATOR STACKED 300 GPUS ACROSS TWO APARTMENTS IN THE SAME BUILDING AND RUNS A $48K/MONTH AI INFERENCE FARM ON VAST AI FROM HIS LIVING ROOM 00:17 he walks past stacks of GPU boxes, "and probably another 100 GPU boxes in the second apartment, let me know in the comments if you want to see them" he rents 2 units in the same building, one as his living space with 200 GPUs in the bedroom and hallway, the second is dedicated and climate controlled just for the other 100 cards a 300 RTX 4090 setup pulls 135 kilowatts fully loaded, his power bill runs $9,800 a month at $0.10 per kwh, on vast ai the same fleet clears $48,000 in gross monthly rental income he never built this in a warehouse because residential electricity in his city is cheaper than commercial under 150 kw, the split apartment trick keeps him under that ceiling while doubling his rack space the same hardware would have cleared maybe $9,000 a month mining ethereum classic in 2022, vast ai pays 5 times that for AI inference because nobody can ship enough H100s to meet startup demand bookmark this and read the article below

starmex

11,545 görüntüleme • 3 ay önce

THIS FRENCH GUY RUNS OPENCLAW AND HERMES AI AGENTS ON A HOMELAB THAT REPLACED GOOGLE DRIVE AND ICLOUD ENTIRELY ON JUST 10 WATTS The project: host his own local AI agents, OpenClaw and Hermes, ditch cloud storage subscriptions entirely by running his own NAS, and use the whole build to prep for his K8s certification – all running on 4 Raspberry Pis. Today's upgrade: a PoE module. One Ethernet cable now does two jobs – powers the Raspberry Pi AND gives it network access at the same time. No separate power adapters, no cable clutter, no juggling connections during maintenance. Total power draw for all 4 Raspberry Pis combined: max 10 watts. He's not renting cloud storage anymore. He's not paying monthly subscriptions for space he already owns the hardware for. His files, his AI agents, his data – all sitting on hardware in his own home, on a fraction of the power a lightbulb uses. Next up: blocking ads network-wide across his entire home, using one of the automation services running on the same setup. This is what "own your data" actually looks like when someone builds it instead of just talking about it. Bookmark this post. Full build in the video below.
0:45

THIS FRENCH GUY RUNS OPENCLAW AND HERMES AI AGENTS ON A HOMELAB THAT REPLACED GOOGLE DRIVE AND ICLOUD ENTIRELY ON JUST 10 WATTS The project: host his own local AI agents, OpenClaw and Hermes, ditch cloud storage subscriptions entirely by running his own NAS, and use the whole build to prep for his K8s certification – all running on 4 Raspberry Pis. Today's upgrade: a PoE module. One Ethernet cable now does two jobs – powers the Raspberry Pi AND gives it network access at the same time. No separate power adapters, no cable clutter, no juggling connections during maintenance. Total power draw for all 4 Raspberry Pis combined: max 10 watts. He's not renting cloud storage anymore. He's not paying monthly subscriptions for space he already owns the hardware for. His files, his AI agents, his data – all sitting on hardware in his own home, on a fraction of the power a lightbulb uses. Next up: blocking ads network-wide across his entire home, using one of the automation services running on the same setup. This is what "own your data" actually looks like when someone builds it instead of just talking about it. Bookmark this post. Full build in the video below.

SCOTTY BEAM

18,175 görüntüleme • 2 ay önce

100 MAC MINIS ON ONE METAL SHELF. IN THE CLOUD, EACH OF THOSE RENTS FOR $120 TO $500 A MONTH. that clip is a homemade rack: wire shelving packed with mac minis, each running macos, wired into one cluster. why macs and not a normal server? two reasons. you can only build, sign, and notarize ios and mac apps on apple hardware. cloud providers know it and charge for it. and apple silicon sips power and runs near-silent, so you can stack a lot of them on a shelf in a room, not a datacenter. the rent they're escaping: a single mac in the cloud runs ~$120 to $500 a month (macstadium up to aws) one engineer documented saving $4,000+ a month by self-hosting mac minis instead of cloud ci runners a used or new mini is a few hundred dollars once, then a few dollars of electricity what a shelf like this actually runs: ios and mac build farms, render jobs, app and device testing automation, and increasingly local ai inference. own the boxes, rent nothing. the uncomfortable part: the cloud was never the only option. it was the convenient one. the people who did the math bought the shelf. the honest caveat: it's real capex up front, real heat and power, and you become the sysadmin. no rented rack means no one to call at 3am. worth it at scale, overkill for one build a week. no rack rental, no per-hour metering, no fleet you don't own. save this before your cloud invoice renews again.
0:25

100 MAC MINIS ON ONE METAL SHELF. IN THE CLOUD, EACH OF THOSE RENTS FOR $120 TO $500 A MONTH. that clip is a homemade rack: wire shelving packed with mac minis, each running macos, wired into one cluster. why macs and not a normal server? two reasons. you can only build, sign, and notarize ios and mac apps on apple hardware. cloud providers know it and charge for it. and apple silicon sips power and runs near-silent, so you can stack a lot of them on a shelf in a room, not a datacenter. the rent they're escaping: a single mac in the cloud runs ~$120 to $500 a month (macstadium up to aws) one engineer documented saving $4,000+ a month by self-hosting mac minis instead of cloud ci runners a used or new mini is a few hundred dollars once, then a few dollars of electricity what a shelf like this actually runs: ios and mac build farms, render jobs, app and device testing automation, and increasingly local ai inference. own the boxes, rent nothing. the uncomfortable part: the cloud was never the only option. it was the convenient one. the people who did the math bought the shelf. the honest caveat: it's real capex up front, real heat and power, and you become the sysadmin. no rented rack means no one to call at 3am. worth it at scale, overkill for one build a week. no rack rental, no per-hour metering, no fleet you don't own. save this before your cloud invoice renews again.

RetroChainer

21,055 görüntüleme • 2 ay önce

$0.05 PER KWH IN PARAGUAY VS $0.15 IN THE US, A 5,000 GPU FARM NEAR ASUNCION CLEARS $4.7M/MO ON VAST AI WHILE US OPERATORS NET $1.8M ON THE SAME RACK the warehouse in the clip is somewhere outside asuncion, multi story racks running floor to ceiling, the operator built it next to a hydroelectric dam where electricity costs 5 cents per kilowatt hour a US warehouse running the same 5,000 GPU rack burns through $1.6M a month in electricity at $0.15/kWh, the same rack in paraguay burns $530K, the gap is $1M of pure margin every 30 days the racks host RTX 4090s, A100s and H100s on vast ai and clore ai, indie AI startups and image generation services rent them by the hour because aws is rationing GPU access to fortune 500 a 4090 nets the US operator $500 to $1,000 a month, the paraguay operator clears $850 to $1,400 on the same card because power eats half what it does north of the equator bitfarms, riot and core scientific have all expanded into paraguay since 2024, the next 10,000 GPU AI farm coming online in 2026 is more likely to sit near a paraguayan dam than in texas bookmark this and read the article below
0:41

$0.05 PER KWH IN PARAGUAY VS $0.15 IN THE US, A 5,000 GPU FARM NEAR ASUNCION CLEARS $4.7M/MO ON VAST AI WHILE US OPERATORS NET $1.8M ON THE SAME RACK the warehouse in the clip is somewhere outside asuncion, multi story racks running floor to ceiling, the operator built it next to a hydroelectric dam where electricity costs 5 cents per kilowatt hour a US warehouse running the same 5,000 GPU rack burns through $1.6M a month in electricity at $0.15/kWh, the same rack in paraguay burns $530K, the gap is $1M of pure margin every 30 days the racks host RTX 4090s, A100s and H100s on vast ai and clore ai, indie AI startups and image generation services rent them by the hour because aws is rationing GPU access to fortune 500 a 4090 nets the US operator $500 to $1,000 a month, the paraguay operator clears $850 to $1,400 on the same card because power eats half what it does north of the equator bitfarms, riot and core scientific have all expanded into paraguay since 2024, the next 10,000 GPU AI farm coming online in 2026 is more likely to sit near a paraguayan dam than in texas bookmark this and read the article below

starmex

27,045 görüntüleme • 3 ay önce

Destra Edge — $DSYNC From Idea to Reality Yesterday, we introduced Destra Edge. Today, we show you what it means. For too long, AI has lived in data centers. Expensive. Centralized. Controlled by a few. Meanwhile, billions of smartphones sit idle. Powerful GPUs. Doing nothing. That doesn’t make sense. So we’re building something different. Destra Edge turns everyday smartphones into a global AI inference network. No servers. No cloud monopolies. Just real compute at the edge. This video shows the system taking shape — how inference is distributed, how it runs on-device, and how results come together without central control. This is not finished. And that’s the point. You’re watching it being built. Layer by layer. In public. The magic is in the scale. More phones means more power. More power means stronger AI. The network grows simply because people join. No passive mining. No fake activity. Only real GPU inference. Measured. Verified. Rewarded. AI shouldn’t live in warehouses. It should live in your pocket. This is edge-level decentralization. This is permissionless inference. This is Destra Network.
2:02

Destra Edge — $DSYNC From Idea to Reality Yesterday, we introduced Destra Edge. Today, we show you what it means. For too long, AI has lived in data centers. Expensive. Centralized. Controlled by a few. Meanwhile, billions of smartphones sit idle. Powerful GPUs. Doing nothing. That doesn’t make sense. So we’re building something different. Destra Edge turns everyday smartphones into a global AI inference network. No servers. No cloud monopolies. Just real compute at the edge. This video shows the system taking shape — how inference is distributed, how it runs on-device, and how results come together without central control. This is not finished. And that’s the point. You’re watching it being built. Layer by layer. In public. The magic is in the scale. More phones means more power. More power means stronger AI. The network grows simply because people join. No passive mining. No fake activity. Only real GPU inference. Measured. Verified. Rewarded. AI shouldn’t live in warehouses. It should live in your pocket. This is edge-level decentralization. This is permissionless inference. This is Destra Network.

Destra Network

20,489 görüntüleme • 8 ay önce

AMD JUST KILLED THE $200 - A - MONTH AI SUBSCRIPTION - A 65-WATT BOX WITH 128GB OF UNIFIED MEMORY THAT RUNS 70B MODELS LOCALLY. that clip is amd's ryzen ai max+ 395 mini box, held up on stage. the whole pitch is one spec: 128gb of high-speed unified memory. why it matters: cpu, gpu and npu share the same 128gb pool, so a model that used to need a rack of vram just fits. what it actually runs: llama 70b in full precision on the built-in gpu open models up to ~100b parameters, locally, no cloud all of it in a mini pc pulling around 65 watts that's the part that ends the subscription: your prompts, your data, your model, on a box on your desk, not metered by the token. honest line: this runs open models, not gpt-5 or claude opus. for chat, coding help and agents on your own data it's more than enough. for the absolute frontier, the cloud still wins. and it isn't free: the box runs ~$1,500 and up. but that's a few months of a $200 sub, then it's yours forever. no monthly ai bill, no tokens metered, no prompt leaving the room. the uncomfortable part isn't that amd made a fast box. it's that "you need the cloud for ai" was a pricing choice, and unified memory just made it optional. follow RetroChainer for the hardware that turns rent into ownership, and bookmark this before your next ai renewal.
0:41

AMD JUST KILLED THE $200 - A - MONTH AI SUBSCRIPTION - A 65-WATT BOX WITH 128GB OF UNIFIED MEMORY THAT RUNS 70B MODELS LOCALLY. that clip is amd's ryzen ai max+ 395 mini box, held up on stage. the whole pitch is one spec: 128gb of high-speed unified memory. why it matters: cpu, gpu and npu share the same 128gb pool, so a model that used to need a rack of vram just fits. what it actually runs: llama 70b in full precision on the built-in gpu open models up to ~100b parameters, locally, no cloud all of it in a mini pc pulling around 65 watts that's the part that ends the subscription: your prompts, your data, your model, on a box on your desk, not metered by the token. honest line: this runs open models, not gpt-5 or claude opus. for chat, coding help and agents on your own data it's more than enough. for the absolute frontier, the cloud still wins. and it isn't free: the box runs ~$1,500 and up. but that's a few months of a $200 sub, then it's yours forever. no monthly ai bill, no tokens metered, no prompt leaving the room. the uncomfortable part isn't that amd made a fast box. it's that "you need the cloud for ai" was a pricing choice, and unified memory just made it optional. follow RetroChainer for the hardware that turns rent into ownership, and bookmark this before your next ai renewal.

RetroChainer

16,995 görüntüleme • 1 ay önce

$NBIS is becoming one of the biggest winners in the new AI economy because it can deliver compute efficiently at scale. The number that mattered most to me was Nebius seeing four or more customers competing for every GPU it brings online (huge for AI cloud market). Management also said they raised prices again and are still selling out across both old and new GPU generations which is one of the cleanest pricing power signals you can get. This is a $100B company in the making.
0:44

$NBIS is becoming one of the biggest winners in the new AI economy because it can deliver compute efficiently at scale. The number that mattered most to me was Nebius seeing four or more customers competing for every GPU it brings online (huge for AI cloud market). Management also said they raised prices again and are still selling out across both old and new GPU generations which is one of the cleanest pricing power signals you can get. This is a $100B company in the making.

Shay Boloor

589,512 görüntüleme • 4 ay önce

ROBA Compute is officially LIVE. This isn't a feature drop. This is ROBA expanding into the AI infrastructure market - hundreds of billions in scale, and we're staking our claim. Rent GPUs on-demand. Enterprise-grade allocation. Transparent USDC pricing. Zero hidden fees and all on-chain. We're already in talks with enterprise clients for compute at scale. 15% of all compute revenue → straight to $ROBA buybacks & burns. GPU capacity is limited at launch. First come, first served. Over the next weeks we will begin accepting new compute partners to provide more power to the marketplace in addition to ROBA's own in-house GPUs and integrated robotics stack. Platform fees are discounted for a limited time only. The infrastructure layer powering physical AI and beyond starts here.
0:41

ROBA Compute is officially LIVE. This isn't a feature drop. This is ROBA expanding into the AI infrastructure market - hundreds of billions in scale, and we're staking our claim. Rent GPUs on-demand. Enterprise-grade allocation. Transparent USDC pricing. Zero hidden fees and all on-chain. We're already in talks with enterprise clients for compute at scale. 15% of all compute revenue → straight to $ROBA buybacks & burns. GPU capacity is limited at launch. First come, first served. Over the next weeks we will begin accepting new compute partners to provide more power to the marketplace in addition to ROBA's own in-house GPUs and integrated robotics stack. Platform fees are discounted for a limited time only. The infrastructure layer powering physical AI and beyond starts here.

Roba Labs

44,287 görüntüleme • 2 ay önce

Fred Thiel Fred Thiel runs the largest public bitcoin miner. His number: a mining site costs about $1M per megawatt, all in. An AI site runs $10-15M per megawatt before a single GPU goes in. Same land, same power hookup, 10-15x the build cost. That's why Mara is converting, not mining. It already controls over 4GW after the Long Ridge and HIF deals, against 1.1GW of live mining today. The catch: Mara had no hyperscaler relationships and no build capability, so it's leaning on Starwood at 80% loan-to-value. The megawatts are real. The tenant demand is borrowed. Chase Lochmiller Chase Lochmiller of Crusoe makes the same energy-first case - the AI constraint isn't chips, it's power you can actually plug in. The tell for $MARA isn't hash rate anymore. It's how many of those gigawatts turn into signed AI leases. Thiel said it to Natalie Brunell Natalie Brunell ⚡️ on Coin Stories. How Mara funded the pivot and what breaks it:
0:56

Fred Thiel Fred Thiel runs the largest public bitcoin miner. His number: a mining site costs about $1M per megawatt, all in. An AI site runs $10-15M per megawatt before a single GPU goes in. Same land, same power hookup, 10-15x the build cost. That's why Mara is converting, not mining. It already controls over 4GW after the Long Ridge and HIF deals, against 1.1GW of live mining today. The catch: Mara had no hyperscaler relationships and no build capability, so it's leaning on Starwood at 80% loan-to-value. The megawatts are real. The tenant demand is borrowed. Chase Lochmiller Chase Lochmiller of Crusoe makes the same energy-first case - the AI constraint isn't chips, it's power you can actually plug in. The tell for $MARA isn't hash rate anymore. It's how many of those gigawatts turn into signed AI leases. Thiel said it to Natalie Brunell Natalie Brunell ⚡️ on Coin Stories. How Mara funded the pivot and what breaks it:

Podcast Alpha

35,196 görüntüleme • 2 ay önce

A USED $700 RTX 3090 HAS MORE MEMORY AND FAR MORE SPEED THAN THE ~$1,000 A2 GOING INTO THIS SERVER. THE A2 STILL WINS - FOR ONE REASON THAT ISN'T PERFORMANCE. that clip is a dell poweredge r760 - a 2u rack server - getting an nvidia a2 dropped in through a riser. this is the enterprise route to local ai. the rung the desktop ladder skips entirely. the a2, verified: 16gb of memory, single-slot, low-profile, and just 40 to 60 watts it's an inference card, built to sit in a server that has no room or spare power for a real gpu so why not a 3090? a used rtx 3090 gives you 24gb and many times the compute for around $700. the a2 gives 16gb, much slower, for roughly a thousand. on raw local-llm value, the desktop card wins, and it isn't close. the a2's whole reason to exist is the thing you can't see on a spec sheet: it fits. a 2u server has no spare gpu power and no room for a triple-slot furnace. the a2 slips into one low-profile slot on 60 watts and lets an existing server do inference without a rebuild. the honest caveats: the r760 is not a desk machine. loaded, it idles at hundreds of watts, and its fans sound like a hair dryer that never turns off. it belongs in a rack or a closet 16gb is still 16gb. the same 7b-to-32b ceiling as the cheap cards. the enterprise badge doesn't buy you a bigger model what it does buy: ecc memory, redundant power, remote management, hot-swap everything. reliability, not speed so who this is for: someone who already runs a rack and wants to add private inference without touching the power or cooling budget. for anyone starting from a desk, the mac mini or the 3090 wins on every axis that matters. the real point: the "best" local ai box depends entirely on what you already own. on a desk, efficiency wins. in a rack, fit and reliability win. the a2 isn't a bad card - it's a card for a constraint the ladder never mentions. no fastest gpu here, no desk-friendly box, no bigger model than the cheap cards already run. save this before you buy enterprise gear for a desktop job.
0:28

A USED $700 RTX 3090 HAS MORE MEMORY AND FAR MORE SPEED THAN THE ~$1,000 A2 GOING INTO THIS SERVER. THE A2 STILL WINS - FOR ONE REASON THAT ISN'T PERFORMANCE. that clip is a dell poweredge r760 - a 2u rack server - getting an nvidia a2 dropped in through a riser. this is the enterprise route to local ai. the rung the desktop ladder skips entirely. the a2, verified: 16gb of memory, single-slot, low-profile, and just 40 to 60 watts it's an inference card, built to sit in a server that has no room or spare power for a real gpu so why not a 3090? a used rtx 3090 gives you 24gb and many times the compute for around $700. the a2 gives 16gb, much slower, for roughly a thousand. on raw local-llm value, the desktop card wins, and it isn't close. the a2's whole reason to exist is the thing you can't see on a spec sheet: it fits. a 2u server has no spare gpu power and no room for a triple-slot furnace. the a2 slips into one low-profile slot on 60 watts and lets an existing server do inference without a rebuild. the honest caveats: the r760 is not a desk machine. loaded, it idles at hundreds of watts, and its fans sound like a hair dryer that never turns off. it belongs in a rack or a closet 16gb is still 16gb. the same 7b-to-32b ceiling as the cheap cards. the enterprise badge doesn't buy you a bigger model what it does buy: ecc memory, redundant power, remote management, hot-swap everything. reliability, not speed so who this is for: someone who already runs a rack and wants to add private inference without touching the power or cooling budget. for anyone starting from a desk, the mac mini or the 3090 wins on every axis that matters. the real point: the "best" local ai box depends entirely on what you already own. on a desk, efficiency wins. in a rack, fit and reliability win. the a2 isn't a bad card - it's a card for a constraint the ladder never mentions. no fastest gpu here, no desk-friendly box, no bigger model than the cheap cards already run. save this before you buy enterprise gear for a desktop job.

Grimmer

24,287 görüntüleme • 2 ay önce

Demis Hassabis just explained why the real AI bottleneck has nothing to do with training runs. Most people picture the AI arms race as who can build the biggest model. GPT-4 or Gemini Ultra style training runs, a few hundred million in compute, fired once or twice a year. The constraint sits somewhere else. Every time a researcher has a new algorithmic idea, a new architecture, a new training technique, they can't just test it on a laptop. They have to run it at the scale where it would actually be deployed, because ideas that look promising at small scale fall apart completely when you put them into a real system. Every research hypothesis burns significant compute before a single line of production code gets written. At a lab like DeepMind, hundreds of researchers are running hundreds of ideas simultaneously. The demand for experimental compute is continuous. It never stops. Now layer the hardware reality on top. GPU lead times are currently 36 to 52 weeks for data center hardware. Global AI data centers are already drawing 29.6 gigawatts, equivalent to the peak power demand of the entire state of New York, and they still can't meet demand. Companies willing to pay any price can't just buy more compute. They wait in line. The speed of scientific discovery in AI is now gated by hardware availability. The next breakthrough is sitting in a researcher's head right now. Whether it gets validated fast enough to matter depends entirely on whether the compute is there when they need it. The AI race gets won by whoever can run the most experiments per month.
0:47

Demis Hassabis just explained why the real AI bottleneck has nothing to do with training runs. Most people picture the AI arms race as who can build the biggest model. GPT-4 or Gemini Ultra style training runs, a few hundred million in compute, fired once or twice a year. The constraint sits somewhere else. Every time a researcher has a new algorithmic idea, a new architecture, a new training technique, they can't just test it on a laptop. They have to run it at the scale where it would actually be deployed, because ideas that look promising at small scale fall apart completely when you put them into a real system. Every research hypothesis burns significant compute before a single line of production code gets written. At a lab like DeepMind, hundreds of researchers are running hundreds of ideas simultaneously. The demand for experimental compute is continuous. It never stops. Now layer the hardware reality on top. GPU lead times are currently 36 to 52 weeks for data center hardware. Global AI data centers are already drawing 29.6 gigawatts, equivalent to the peak power demand of the entire state of New York, and they still can't meet demand. Companies willing to pay any price can't just buy more compute. They wait in line. The speed of scientific discovery in AI is now gated by hardware availability. The next breakthrough is sitting in a researcher's head right now. Whether it gets validated fast enough to matter depends entirely on whether the compute is there when they need it. The AI race gets won by whoever can run the most experiments per month.

Aakash Gupta

32,150 görüntüleme • 5 ay önce

A FARM OF 90 RTX 3090s BUILT FOR ETHEREUM MINING IN 2021 IS NOW SERVING LOCAL LLM INFERENCE FROM THE SAME GARAGE. THE PART THAT MATTERS ISN'T THE SIZE, IT'S THAT THE OWNER NEVER UNPLUGGED THE RIG, HE JUST CHANGED THE WORKLOAD the camera walks through the room. three custom wood frames stacked floor to ceiling. ninety 3090s breathing rainbow LEDs in unison. a yellow industrial duct ripped through the wall to pull the heat out. each card pulling 250W under load, the whole room drawing 22kW continuous. this is not a hobby setup. it is a small datacenter dressed up as a bedroom do the math. 90 cards times 24GB VRAM is 2,160GB of pooled memory, enough to load every frontier open weight model at once. at $700 used per card, each 3090 pays for itself against a Claude Code Max subscription in about three months. the same RTX 3090 i wrote about last week as the best value per dollar for a local AI rig, stacked 90 times in a garage this is not a crypto relic. it is the new datacenter, built from cards that gamers retired in 2022 and crypto miners gave up on in 2023, now quietly answering prompts that OpenAI charges you $200 a month for
0:33

A FARM OF 90 RTX 3090s BUILT FOR ETHEREUM MINING IN 2021 IS NOW SERVING LOCAL LLM INFERENCE FROM THE SAME GARAGE. THE PART THAT MATTERS ISN'T THE SIZE, IT'S THAT THE OWNER NEVER UNPLUGGED THE RIG, HE JUST CHANGED THE WORKLOAD the camera walks through the room. three custom wood frames stacked floor to ceiling. ninety 3090s breathing rainbow LEDs in unison. a yellow industrial duct ripped through the wall to pull the heat out. each card pulling 250W under load, the whole room drawing 22kW continuous. this is not a hobby setup. it is a small datacenter dressed up as a bedroom do the math. 90 cards times 24GB VRAM is 2,160GB of pooled memory, enough to load every frontier open weight model at once. at $700 used per card, each 3090 pays for itself against a Claude Code Max subscription in about three months. the same RTX 3090 i wrote about last week as the best value per dollar for a local AI rig, stacked 90 times in a garage this is not a crypto relic. it is the new datacenter, built from cards that gamers retired in 2022 and crypto miners gave up on in 2023, now quietly answering prompts that OpenAI charges you $200 a month for

Ridark

64,499 görüntüleme • 3 ay önce

BOLT A GPU ONTO THIS $200 FANLESS BOARD AND IT DOES THE TWO THINGS A RASPBERRY PI PHYSICALLY CAN'T: LOCAL AI AND 4K TRANSCODING. that clip is a zimaboard with a graphics card plugged straight into its pcie slot. that slot is the whole point. a pi has gpio and usb, but no real pcie, so it tops out where this one starts. drop a gpu in and the tiny box suddenly does: ollama and local models with actual gpu acceleration, not cpu crawl plex or jellyfin transcoding several 4k streams at once, in hardware anything else that was "you need a real pc for that", on a fanless board the rest of the board backs it up: quad-core intel up to 3.6, dual sata, dual 2.5gbe, runs zimaos, linux, even windows. so one silent box becomes a nas, a media server, and a local ai node at the same time. no tower, no raspberry-pi ceiling, no cloud gpu rented by the hour. the uncomfortable part isn't that it takes a gpu. it's that "a pi can do it" quietly stopped being true the moment the workload needed a real pcie lane. follow RetroChainer for the small boxes that punch way above their size, and bookmark this before you buy a whole tower for one gpu job.
0:21

BOLT A GPU ONTO THIS $200 FANLESS BOARD AND IT DOES THE TWO THINGS A RASPBERRY PI PHYSICALLY CAN'T: LOCAL AI AND 4K TRANSCODING. that clip is a zimaboard with a graphics card plugged straight into its pcie slot. that slot is the whole point. a pi has gpio and usb, but no real pcie, so it tops out where this one starts. drop a gpu in and the tiny box suddenly does: ollama and local models with actual gpu acceleration, not cpu crawl plex or jellyfin transcoding several 4k streams at once, in hardware anything else that was "you need a real pc for that", on a fanless board the rest of the board backs it up: quad-core intel up to 3.6, dual sata, dual 2.5gbe, runs zimaos, linux, even windows. so one silent box becomes a nas, a media server, and a local ai node at the same time. no tower, no raspberry-pi ceiling, no cloud gpu rented by the hour. the uncomfortable part isn't that it takes a gpu. it's that "a pi can do it" quietly stopped being true the moment the workload needed a real pcie lane. follow RetroChainer for the small boxes that punch way above their size, and bookmark this before you buy a whole tower for one gpu job.

RetroChainer

24,813 görüntüleme • 1 ay önce

THE MS-A2 IS THE MINISFORUM BOX HERMES AGENT WAS NEVER MEANT TO RUN ON, AND THAT'S BY DESIGN The MS-S1 MAX earns its spot running Hermes Agent because of one thing. 128GB of unified memory, up to 96GB of it handed straight to the GPU. That's the only reason a 120B model fits and runs locally for $0 a month The MS-A2 solves a different problem. Ryzen 9 9955HX, 16 cores, 32 threads, up to 96GB of regular DDR5-5600, no unified pool Three M.2 PCIe 4.0 slots, one U.2, two 22110. Dual 10Gbps SFP+ LAN plus 2.5G. WiFi 6E. Bluetooth 5.3. A slide-out motherboard for fast upgrades. A real PCIe x16 slot that actually takes a low-profile GPU That last part is where the two machines split for good. The MS-S1 MAX's PCIe slot won't take a GPU at all, every bit of GPU power has to come from the unified chip itself. The MS-A2 trades that unified memory trick for raw expandability instead One box runs a local AI agent. The other runs a home lab that needs storage, networking, and room to grow. Minisforum built both on purpose, not as the same product wearing two names
1:06

THE MS-A2 IS THE MINISFORUM BOX HERMES AGENT WAS NEVER MEANT TO RUN ON, AND THAT'S BY DESIGN The MS-S1 MAX earns its spot running Hermes Agent because of one thing. 128GB of unified memory, up to 96GB of it handed straight to the GPU. That's the only reason a 120B model fits and runs locally for $0 a month The MS-A2 solves a different problem. Ryzen 9 9955HX, 16 cores, 32 threads, up to 96GB of regular DDR5-5600, no unified pool Three M.2 PCIe 4.0 slots, one U.2, two 22110. Dual 10Gbps SFP+ LAN plus 2.5G. WiFi 6E. Bluetooth 5.3. A slide-out motherboard for fast upgrades. A real PCIe x16 slot that actually takes a low-profile GPU That last part is where the two machines split for good. The MS-S1 MAX's PCIe slot won't take a GPU at all, every bit of GPU power has to come from the unified chip itself. The MS-A2 trades that unified memory trick for raw expandability instead One box runs a local AI agent. The other runs a home lab that needs storage, networking, and room to grow. Minisforum built both on purpose, not as the same product wearing two names

NO1ennn

25,645 görüntüleme • 3 ay önce

Everything in the economy runs on the same input. Compute, AI, robotics, manufacturing, transport... it's all energy underneath. That matters now because of demographics. Fewer workers every year means the economy only grows if output per head grows fast, and the engine for that is the Exponential Age... AI and robotics doing the work people used to. But all of that runs on energy. So the cost of energy is the ceiling on the whole thing. That's why the EU, Japan, China, the US and the UK are all moving at the same time. If we can take the cost of energy to near zero, GDP per capita stops being hostage to the price of a barrel of oil. That's the way out of this mess.
0:50

Everything in the economy runs on the same input. Compute, AI, robotics, manufacturing, transport... it's all energy underneath. That matters now because of demographics. Fewer workers every year means the economy only grows if output per head grows fast, and the engine for that is the Exponential Age... AI and robotics doing the work people used to. But all of that runs on energy. So the cost of energy is the ceiling on the whole thing. That's why the EU, Japan, China, the US and the UK are all moving at the same time. If we can take the cost of energy to near zero, GDP per capita stops being hostage to the price of a barrel of oil. That's the way out of this mess.

Raoul Pal

122,479 görüntüleme • 2 ay önce

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.
0:42

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

Akshay 🚀

343,270 görüntüleme • 1 ay önce

🎙️ Want a full breakdown on why cost per token is a key metric for AI infrastructure TCO? We’re operating in a power-constrained world. Demand for intelligence is scaling faster than infrastructure can keep up—so metrics like cost per GPU hour or FLOPs per dollar don’t tell the full story. When it comes to TCO, Cost per Token is the metric that captures end-to-end performance across GPUs, CPUs, networking, software, and ecosystem making it the key driver of real profitability and scalability in AI. If you’re not measuring efficiency at the token level, you’re missing what actually drives cost. Watch the deep-dive conversation with Dr. Gerro Prinsloo , Nader Khalil and Carter Abdallah, live on the NVIDIA YouTube channel.
1:05

🎙️ Want a full breakdown on why cost per token is a key metric for AI infrastructure TCO? We’re operating in a power-constrained world. Demand for intelligence is scaling faster than infrastructure can keep up—so metrics like cost per GPU hour or FLOPs per dollar don’t tell the full story. When it comes to TCO, Cost per Token is the metric that captures end-to-end performance across GPUs, CPUs, networking, software, and ecosystem making it the key driver of real profitability and scalability in AI. If you’re not measuring efficiency at the token level, you’re missing what actually drives cost. Watch the deep-dive conversation with Dr. Gerro Prinsloo , Nader Khalil and Carter Abdallah, live on the NVIDIA YouTube channel.

NVIDIA

21,970 görüntüleme • 5 ay önce

this is what 12 gigs of VRAM built in 2026. a 9 billion parameter model running on a 5 year old RTX 3060 wrote a full space shooter from a single prompt. blank screen on first try. i came back with a bug list and the same model on the same card fixed every issue across 11 files without touching a single line myself. enemies still looked wrong so i pushed another iteration and now the game has pixel art octopi, particle effects, screen shake, projectile physics and a combo system. all running locally on a card that was designed to play fortnite. three iterations. zero cloud. zero API calls. every token generated on hardware sitting under my desk. the model reads its own code, finds what's broken, patches it, validates syntax and restarts the server. i just describe what's wrong and it handles the rest. people are paying monthly subscriptions to type into a browser tab and wait for a server farm to respond. meanwhile a GPU you can find used on ebay is running a full autonomous hermes agent framework with 31 tools, 128K context window and thinking mode generating at 29 tokens per second nonstop. the game still needs work. level upgrades don't trigger and boss fights need tuning. but the fact that i'm iterating on gameplay balance instead of debugging whether the code runs at all tells you where this is headed. every iteration the game gets better on the same hardware. same 12 gigs. same 9 billion parameters. same RTX 3060 from 5 years ago your GPU is not a gaming card anymore. it's a local AI lab that never sends your data anywhere.
0:28

this is what 12 gigs of VRAM built in 2026. a 9 billion parameter model running on a 5 year old RTX 3060 wrote a full space shooter from a single prompt. blank screen on first try. i came back with a bug list and the same model on the same card fixed every issue across 11 files without touching a single line myself. enemies still looked wrong so i pushed another iteration and now the game has pixel art octopi, particle effects, screen shake, projectile physics and a combo system. all running locally on a card that was designed to play fortnite. three iterations. zero cloud. zero API calls. every token generated on hardware sitting under my desk. the model reads its own code, finds what's broken, patches it, validates syntax and restarts the server. i just describe what's wrong and it handles the rest. people are paying monthly subscriptions to type into a browser tab and wait for a server farm to respond. meanwhile a GPU you can find used on ebay is running a full autonomous hermes agent framework with 31 tools, 128K context window and thinking mode generating at 29 tokens per second nonstop. the game still needs work. level upgrades don't trigger and boss fights need tuning. but the fact that i'm iterating on gameplay balance instead of debugging whether the code runs at all tells you where this is headed. every iteration the game gets better on the same hardware. same 12 gigs. same 9 billion parameters. same RTX 3060 from 5 years ago your GPU is not a gaming card anymore. it's a local AI lab that never sends your data anywhere.

Sudo su

170,848 görüntüleme • 6 ay önce