正在加载视频...

视频加载失败

返回首页

A $40 BC250 BOARD WITH 16GB GDDR6 GETS A GITHUB FIRMWARE FLASH, THE DEFAULT 8/8 CPU/GPU SPLIT REWRITES TO 0.5/15.5 AND OPENS 15.5GB OF UNIFIED MEMORY TO OLLAMA, DEAD CRYPTO HARDWARE JUST DOUBLED ITS AI CEILING 01:08 the operator points at the chart, "you only give 512 megs to...

antisadh's profile picture

Antid

1,440 subscribers

25,619 次观看 • 3 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

SONY SHIPS THIS APU IN A $200 PS5. BITCOIN OPERATIONS DUMPED IT ON EBAY FOR $40. HE 3D PRINTED A CASE OVERNIGHT AND KILLED $439 A MONTH IN ANTHROPIC AND OPENAI BILLS 00:34 he turns the board over and says "this was actually meant for bitcoin mining originally, and this uses ps5 adjacent hardware for the apu" $80 bc250 off ebay -> one 8 pin connector, no pcie, no motherboard -> $50 flex itx 500w psu -> 20 hour ASA print for the case -> blow four years of mining dust out with an air compressor -> 16gb gddr6 unified, ollama up in 4 minutes sony sold 117 million ps5s on this silicon, bitcoin operations bought the mining variant by the pallet, ethereum's merge stranded all of it, and now the same oberon apu that renders spider-man runs a local coding model for the price of a dinner the honest part: it arrived filthy, the compressor barely touched the dust, and you need a 3d printer before the board does anything at all $80 amd bc250 + $50 flex itx 500w + 20h ASA print + oberon apu with 16gb gddr6 + ubuntu + ollama watch and save it, then kill your $439 stack this weekend, kimi does the same work at $19 flat and the whole map is in the article below
1:47

SONY SHIPS THIS APU IN A $200 PS5. BITCOIN OPERATIONS DUMPED IT ON EBAY FOR $40. HE 3D PRINTED A CASE OVERNIGHT AND KILLED $439 A MONTH IN ANTHROPIC AND OPENAI BILLS 00:34 he turns the board over and says "this was actually meant for bitcoin mining originally, and this uses ps5 adjacent hardware for the apu" $80 bc250 off ebay -> one 8 pin connector, no pcie, no motherboard -> $50 flex itx 500w psu -> 20 hour ASA print for the case -> blow four years of mining dust out with an air compressor -> 16gb gddr6 unified, ollama up in 4 minutes sony sold 117 million ps5s on this silicon, bitcoin operations bought the mining variant by the pallet, ethereum's merge stranded all of it, and now the same oberon apu that renders spider-man runs a local coding model for the price of a dinner the honest part: it arrived filthy, the compressor barely touched the dust, and you need a 3d printer before the board does anything at all $80 amd bc250 + $50 flex itx 500w + 20h ASA print + oberon apu with 16gb gddr6 + ubuntu + ollama watch and save it, then kill your $439 stack this weekend, kimi does the same work at $19 flat and the whole map is in the article below

Antid

35,879 次观看 • 1 个月前

7-YEAR-OLD RASPBERRY PI WITH 2GB OF RAM IS RUNNING DEEPSEEK RIGHT NOW WHILE SAM ALTMAN BILLS $200/MO AND DARIO AMODEI BILLS $200/MO FOR A MODEL YOU NEVER GET TO KEEP 00:34 he points at the smallest board on the bench and says "out of all these I'm genuinely shocked that I actually got it up and running on this raspberry pi 4b with 2 gigs of ram" raspberry pi 4b, 2gb, seven years old -> ollama with deepseek r1 1.5b -> a dell micro form factor doing cpu-only inference next to it -> an m1 max macbook with 64gb unified memory beating the dual gpu tower on large models -> four machines, four architectures, all serving locally apple's unified memory puts the ram on the same die as the cpu and gpu, so a laptop outruns a two-card workstation on big models, and the cheapest board on that bench costs less than four days of a claude max seat the honest part: 2gb is the actual floor and he says it was rough to get there, a 4gb or 8gb pi would have taken minutes instead, and 1.5b parameters is a model you use for structure not for answers raspberry pi 4b 2gb + dell micro form factor + m1 max 64gb + dual gpu tower + ollama + deepseek r1 watch and save it, musk gave away the agent and moonshot gave away the weights, the full $400 to $25 routing table is in the article below
3:16

7-YEAR-OLD RASPBERRY PI WITH 2GB OF RAM IS RUNNING DEEPSEEK RIGHT NOW WHILE SAM ALTMAN BILLS $200/MO AND DARIO AMODEI BILLS $200/MO FOR A MODEL YOU NEVER GET TO KEEP 00:34 he points at the smallest board on the bench and says "out of all these I'm genuinely shocked that I actually got it up and running on this raspberry pi 4b with 2 gigs of ram" raspberry pi 4b, 2gb, seven years old -> ollama with deepseek r1 1.5b -> a dell micro form factor doing cpu-only inference next to it -> an m1 max macbook with 64gb unified memory beating the dual gpu tower on large models -> four machines, four architectures, all serving locally apple's unified memory puts the ram on the same die as the cpu and gpu, so a laptop outruns a two-card workstation on big models, and the cheapest board on that bench costs less than four days of a claude max seat the honest part: 2gb is the actual floor and he says it was rough to get there, a 4gb or 8gb pi would have taken minutes instead, and 1.5b parameters is a model you use for structure not for answers raspberry pi 4b 2gb + dell micro form factor + m1 max 64gb + dual gpu tower + ollama + deepseek r1 watch and save it, musk gave away the agent and moonshot gave away the weights, the full $400 to $25 routing table is in the article below

Antid

10,442 次观看 • 1 个月前

BOLT A GPU ONTO THIS $200 FANLESS BOARD AND IT DOES THE TWO THINGS A RASPBERRY PI PHYSICALLY CAN'T: LOCAL AI AND 4K TRANSCODING. that clip is a zimaboard with a graphics card plugged straight into its pcie slot. that slot is the whole point. a pi has gpio and usb, but no real pcie, so it tops out where this one starts. drop a gpu in and the tiny box suddenly does: ollama and local models with actual gpu acceleration, not cpu crawl plex or jellyfin transcoding several 4k streams at once, in hardware anything else that was "you need a real pc for that", on a fanless board the rest of the board backs it up: quad-core intel up to 3.6, dual sata, dual 2.5gbe, runs zimaos, linux, even windows. so one silent box becomes a nas, a media server, and a local ai node at the same time. no tower, no raspberry-pi ceiling, no cloud gpu rented by the hour. the uncomfortable part isn't that it takes a gpu. it's that "a pi can do it" quietly stopped being true the moment the workload needed a real pcie lane. follow RetroChainer for the small boxes that punch way above their size, and bookmark this before you buy a whole tower for one gpu job.
0:21

BOLT A GPU ONTO THIS $200 FANLESS BOARD AND IT DOES THE TWO THINGS A RASPBERRY PI PHYSICALLY CAN'T: LOCAL AI AND 4K TRANSCODING. that clip is a zimaboard with a graphics card plugged straight into its pcie slot. that slot is the whole point. a pi has gpio and usb, but no real pcie, so it tops out where this one starts. drop a gpu in and the tiny box suddenly does: ollama and local models with actual gpu acceleration, not cpu crawl plex or jellyfin transcoding several 4k streams at once, in hardware anything else that was "you need a real pc for that", on a fanless board the rest of the board backs it up: quad-core intel up to 3.6, dual sata, dual 2.5gbe, runs zimaos, linux, even windows. so one silent box becomes a nas, a media server, and a local ai node at the same time. no tower, no raspberry-pi ceiling, no cloud gpu rented by the hour. the uncomfortable part isn't that it takes a gpu. it's that "a pi can do it" quietly stopped being true the moment the workload needed a real pcie lane. follow RetroChainer for the small boxes that punch way above their size, and bookmark this before you buy a whole tower for one gpu job.

RetroChainer

24,813 次观看 • 1 个月前

ALIBABA GAVE QWEN AWAY FREE, ZUCKERBERG GAVE LLAMA AWAY FREE, AND JENSEN HUANG STILL CHARGES $300 FOR THE 12GB THAT RUNS THEM. THE $80 AMD BOARD HAS 16GB. HE IS TESTING BOTH THIS WEEK AGAINST $200/MO CLAUDE CODE MAX AND $200/MO CHATGPT PRO 00:19 he points at a dead gtx 680 he keeps on the shelf purely for decoration and says "i'm actually really gpu poor" $300 rtx 3060 with 12gb -> $80 bc250 with 16gb -> ubuntu server on both -> 8gb tier, 12 to 16 tier, 24gb ceiling on a four year old 3090 -> qwen against llama at every tier -> tokens per dollar published jensen shipped 12gb on the 3060 in 2021 and consumer vram has barely moved since, alibaba and meta gave the weights away for nothing, the models are free and the only thing left to buy is memory the honest part: he says it himself, this is apples and oranges, and there is not a single benchmark number in this video yet $80 amd bc250 with 16gb gddr6 + $300 rtx 3060 with 12gb + a four year old 3090 with 24gb + ubuntu server + ollama + qwen + llama watch and save it, then kill your $439 stack this weekend, kimi does the same work at $19 flat and the whole map is in the article below
2:28

ALIBABA GAVE QWEN AWAY FREE, ZUCKERBERG GAVE LLAMA AWAY FREE, AND JENSEN HUANG STILL CHARGES $300 FOR THE 12GB THAT RUNS THEM. THE $80 AMD BOARD HAS 16GB. HE IS TESTING BOTH THIS WEEK AGAINST $200/MO CLAUDE CODE MAX AND $200/MO CHATGPT PRO 00:19 he points at a dead gtx 680 he keeps on the shelf purely for decoration and says "i'm actually really gpu poor" $300 rtx 3060 with 12gb -> $80 bc250 with 16gb -> ubuntu server on both -> 8gb tier, 12 to 16 tier, 24gb ceiling on a four year old 3090 -> qwen against llama at every tier -> tokens per dollar published jensen shipped 12gb on the 3060 in 2021 and consumer vram has barely moved since, alibaba and meta gave the weights away for nothing, the models are free and the only thing left to buy is memory the honest part: he says it himself, this is apples and oranges, and there is not a single benchmark number in this video yet $80 amd bc250 with 16gb gddr6 + $300 rtx 3060 with 12gb + a four year old 3090 with 24gb + ubuntu server + ollama + qwen + llama watch and save it, then kill your $439 stack this weekend, kimi does the same work at $19 flat and the whole map is in the article below

Antid

13,974 次观看 • 1 个月前

A $351 MINI PC IS RUNNING 26-BILLION-PARAMETER AI MODELS AT 20 TOKENS/SEC AND HERMES AGENT ON TOP OF IT This is the Minisforum UM790 Pro. AMD Ryzen 9 7940HS, Radeon 780M iGPU, 48GB DDR5. The BIOS reports the GPU has 4GB of VRAM Here's the part people get wrong. The 780M has no dedicated VRAM at all it borrows from system RAM. Vulkan ignores the BIOS number and reads the full 48GB pool directly That's the whole trick. 21+ GB allocated to model weights on a "4GB" GPU. Because the models are MoE, only 4 billion of the parameters activate per token a fraction of the reads a dense model needs. That's why 20 tok/s works here Gemma 4 26B MoE holds 19.5 tok/s with 196K context. Qwen3.5-35B-A3B holds 20.8. Nemotron Cascade 2 clears 24.8. A dense 31B, by contrast, drops to 4 tok/s it reads the entire model every token, no way around it On top: Hermes Agent, full agentic workflows terminal, file ops, web, 40+ tools against local models only. No API keys. The wall between you and a usable local agent used to be a GPU you couldn't afford. Now it's a BIOS setting most people never check Bookmark this & Try it yourself ↓
1:22

A $351 MINI PC IS RUNNING 26-BILLION-PARAMETER AI MODELS AT 20 TOKENS/SEC AND HERMES AGENT ON TOP OF IT This is the Minisforum UM790 Pro. AMD Ryzen 9 7940HS, Radeon 780M iGPU, 48GB DDR5. The BIOS reports the GPU has 4GB of VRAM Here's the part people get wrong. The 780M has no dedicated VRAM at all it borrows from system RAM. Vulkan ignores the BIOS number and reads the full 48GB pool directly That's the whole trick. 21+ GB allocated to model weights on a "4GB" GPU. Because the models are MoE, only 4 billion of the parameters activate per token a fraction of the reads a dense model needs. That's why 20 tok/s works here Gemma 4 26B MoE holds 19.5 tok/s with 196K context. Qwen3.5-35B-A3B holds 20.8. Nemotron Cascade 2 clears 24.8. A dense 31B, by contrast, drops to 4 tok/s it reads the entire model every token, no way around it On top: Hermes Agent, full agentic workflows terminal, file ops, web, 40+ tools against local models only. No API keys. The wall between you and a usable local agent used to be a GPU you couldn't afford. Now it's a BIOS setting most people never check Bookmark this & Try it yourself ↓

slash1s

35,742 次观看 • 3 个月前

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.
2:24

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.

Grummz

30,573 次观看 • 4 个月前

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.
0:42

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

Akshay 🚀

344,670 次观看 • 1 个月前

THE MS-A2 IS THE MINISFORUM BOX HERMES AGENT WAS NEVER MEANT TO RUN ON, AND THAT'S BY DESIGN The MS-S1 MAX earns its spot running Hermes Agent because of one thing. 128GB of unified memory, up to 96GB of it handed straight to the GPU. That's the only reason a 120B model fits and runs locally for $0 a month The MS-A2 solves a different problem. Ryzen 9 9955HX, 16 cores, 32 threads, up to 96GB of regular DDR5-5600, no unified pool Three M.2 PCIe 4.0 slots, one U.2, two 22110. Dual 10Gbps SFP+ LAN plus 2.5G. WiFi 6E. Bluetooth 5.3. A slide-out motherboard for fast upgrades. A real PCIe x16 slot that actually takes a low-profile GPU That last part is where the two machines split for good. The MS-S1 MAX's PCIe slot won't take a GPU at all, every bit of GPU power has to come from the unified chip itself. The MS-A2 trades that unified memory trick for raw expandability instead One box runs a local AI agent. The other runs a home lab that needs storage, networking, and room to grow. Minisforum built both on purpose, not as the same product wearing two names
1:06

THE MS-A2 IS THE MINISFORUM BOX HERMES AGENT WAS NEVER MEANT TO RUN ON, AND THAT'S BY DESIGN The MS-S1 MAX earns its spot running Hermes Agent because of one thing. 128GB of unified memory, up to 96GB of it handed straight to the GPU. That's the only reason a 120B model fits and runs locally for $0 a month The MS-A2 solves a different problem. Ryzen 9 9955HX, 16 cores, 32 threads, up to 96GB of regular DDR5-5600, no unified pool Three M.2 PCIe 4.0 slots, one U.2, two 22110. Dual 10Gbps SFP+ LAN plus 2.5G. WiFi 6E. Bluetooth 5.3. A slide-out motherboard for fast upgrades. A real PCIe x16 slot that actually takes a low-profile GPU That last part is where the two machines split for good. The MS-S1 MAX's PCIe slot won't take a GPU at all, every bit of GPU power has to come from the unified chip itself. The MS-A2 trades that unified memory trick for raw expandability instead One box runs a local AI agent. The other runs a home lab that needs storage, networking, and room to grow. Minisforum built both on purpose, not as the same product wearing two names

NO1ennn

25,645 次观看 • 3 个月前

NVIDIA WANTS $2,000 FOR 16GB OF VRAM. SONY'S PS5 CHIP DOES IT FOR $80 AND BITCOIN OPERATIONS ARE LIQUIDATING THEM FASTER THAN EBAY CAN LIST THEM, THE PRICE JUMPED 50% IN THREE WEEKS 01:01 he holds it up to the camera and says "these were originally produced for mining bitcoin, but now a lot of bitcoin operations are starting to sell these off" $80 bc250 off ebay -> a slimmed down ps5 apu with 16gb of shared gddr6 -> m.2 slot, sata, dual ethernet, displayport, all on the card -> $50 flex itx psu -> 3d print the case, then 3d print a second tool just to pry the heatsink fins open -> clip a fan onto it -> ubuntu and ollama in one curl line nvidia charges $2,000 for a card with this much vram and sony put the same class of silicon in 117 million consoles, then bitcoin operators bought the mining variant by the pallet and started dumping it, the listings went from $80 to $120 in three weeks and they are still moving the honest part: these shipped as rack cards with server fans screaming through a row of them, pull one out alone and it has no cooling story at all, the entire diy scene exists because this hardware was never meant to live outside a rack amd bc250 + 16gb shared gddr6 + flex itx psu + 3d printed case and fin tool + ubuntu + ollama watch and save it, elon open sourced grok's body and moonshot open sourced kimi's brain, the full $600 to $14 build is in the article below
2:18

NVIDIA WANTS $2,000 FOR 16GB OF VRAM. SONY'S PS5 CHIP DOES IT FOR $80 AND BITCOIN OPERATIONS ARE LIQUIDATING THEM FASTER THAN EBAY CAN LIST THEM, THE PRICE JUMPED 50% IN THREE WEEKS 01:01 he holds it up to the camera and says "these were originally produced for mining bitcoin, but now a lot of bitcoin operations are starting to sell these off" $80 bc250 off ebay -> a slimmed down ps5 apu with 16gb of shared gddr6 -> m.2 slot, sata, dual ethernet, displayport, all on the card -> $50 flex itx psu -> 3d print the case, then 3d print a second tool just to pry the heatsink fins open -> clip a fan onto it -> ubuntu and ollama in one curl line nvidia charges $2,000 for a card with this much vram and sony put the same class of silicon in 117 million consoles, then bitcoin operators bought the mining variant by the pallet and started dumping it, the listings went from $80 to $120 in three weeks and they are still moving the honest part: these shipped as rack cards with server fans screaming through a row of them, pull one out alone and it has no cooling story at all, the entire diy scene exists because this hardware was never meant to live outside a rack amd bc250 + 16gb shared gddr6 + flex itx psu + 3d printed case and fin tool + ubuntu + ollama watch and save it, elon open sourced grok's body and moonshot open sourced kimi's brain, the full $600 to $14 build is in the article below

Antid

59,393 次观看 • 1 个月前

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!
0:56

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,756 次观看 • 2 年前

RTX 3090 was never designed to be a server GPU. Yet someone just built a 4-GPU AI workstation around four of them. 4× RTX 3090 96GB VRAM 256GB ECC RAM AMD EPYC 7K62 2000W PSU 11 fans The interesting part isn't the performance. It's the memory. Four 24GB cards can be found for a fraction of what you'd pay for modern datacenter GPUs with similar total VRAM. The catch is obvious: 96GB doesn't behave like one 96GB GPU. You still have to deal with GPU-to-GPU communication, power and heat. But for local AI, this is exactly why the 3090 refuses to disappear. Would you build this today?
0:51

RTX 3090 was never designed to be a server GPU. Yet someone just built a 4-GPU AI workstation around four of them. 4× RTX 3090 96GB VRAM 256GB ECC RAM AMD EPYC 7K62 2000W PSU 11 fans The interesting part isn't the performance. It's the memory. Four 24GB cards can be found for a fraction of what you'd pay for modern datacenter GPUs with similar total VRAM. The catch is obvious: 96GB doesn't behave like one 96GB GPU. You still have to deal with GPU-to-GPU communication, power and heat. But for local AI, this is exactly why the 3090 refuses to disappear. Would you build this today?

KyzoroX

36,552 次观看 • 1 个月前

THIS $1,300 CARD HAS THE SAME 32GB OF VRAM AS A $2,000+ RTX 5090 - FOR LOCAL AI, THAT'S THE SPEC THAT DECIDES EVERYTHING. that clip is the gigabyte radeon ai pro r9700 ai top, 32gb of vram, built for local ai. the pitch is simple: the same memory capacity as an rtx 5090, for roughly a third of what a 5090 actually sells for. here's why that matters, and where it doesn't. for ai, vram is the gate. a model either fits in memory or it doesn't. 32gb loads models a 16gb card can't touch, full stop. so for running local llms and image models, this card does the one thing that counts, cheap. honest line: this is not a 5090 for gaming. the 5090 is far faster at raster and ray tracing. they only match on how much vram they carry. if you want frames, buy the 5090. if you want to fit big models locally, you just found the same 32gb for a third of the money. no $2,000+ price tag, no cloud gpu rental, no model too big for 16gb. the uncomfortable part isn't that amd is cheaper. it's that for local ai you were about to overpay for gaming power you'd never use. follow RetroChainer for more on paying for the spec that matters, and bookmark this before your next gpu.
1:52

THIS $1,300 CARD HAS THE SAME 32GB OF VRAM AS A $2,000+ RTX 5090 - FOR LOCAL AI, THAT'S THE SPEC THAT DECIDES EVERYTHING. that clip is the gigabyte radeon ai pro r9700 ai top, 32gb of vram, built for local ai. the pitch is simple: the same memory capacity as an rtx 5090, for roughly a third of what a 5090 actually sells for. here's why that matters, and where it doesn't. for ai, vram is the gate. a model either fits in memory or it doesn't. 32gb loads models a 16gb card can't touch, full stop. so for running local llms and image models, this card does the one thing that counts, cheap. honest line: this is not a 5090 for gaming. the 5090 is far faster at raster and ray tracing. they only match on how much vram they carry. if you want frames, buy the 5090. if you want to fit big models locally, you just found the same 32gb for a third of the money. no $2,000+ price tag, no cloud gpu rental, no model too big for 16gb. the uncomfortable part isn't that amd is cheaper. it's that for local ai you were about to overpay for gaming power you'd never use. follow RetroChainer for more on paying for the spec that matters, and bookmark this before your next gpu.

RetroChainer

38,844 次观看 • 2 个月前

Jensen Huang just identified the next $200 billion market (Save this). The shift starts with a observation about agentic AI that changes everything about infrastructure. In the era of training and inference, the GPU was everything while CPU was a traffic cop, scheduling work, managing memory, dispatching tasks while the GPU did the heavy lifting. Agentic AI breaks that model entirely. An AI agent does not just run a single inference pass but rather it plans, calls tools, executes code in sandboxes, retrieves data from multiple sources and loops through complex multi-step reasoning sequences often thousands of times per second at scale. Every one of those operations runs through the CPU and the GPU sits idle waiting for the CPU to prepare the next task, supply the right context and execute the retrieval and tool calling logic fast enough to keep the accelerators fed. The CPU is now the conductor and the GPU is the orchestra and the bottleneck is the conductor falling behind. This is showing up in production AI factory utilization right now, which is exactly why Jensen built Vera from scratch rather than licensing x86. Vera achieves 40% lower peak memory latency than x86, 50% faster core to core communication, and 1.8 times the agentic sandbox performance of current x86 processors on a purpose-built architecture designed around the agentic loop. Now here is where the investment thesis gets interesting. The obvious beneficiary is Nvidia itself, and that thesis is real. Nvidia's CFO has guided for nearly $20 billion in Vera CPU revenue this fiscal year alone, a market Nvidia had zero presence in just three years ago. Intel held 60% of server CPU market share as recently as Q4 2025 and that transition is now happening at a pace Intel structurally cannot respond to. But the deeper question is, what architecture is Vera actually built on? Vera's Olympus cores are ARM compatible and every single Vera CPU deployed in every Vera Rubin rack in every data center in the world runs on ARM architecture. And ARM Holdings collects a royalty on every one of them. ARM does not make chips but rather licenses the instruction set architecture and CPU core designs that others build on top of. Every time Nvidia ships a Vera CPU, every time a hyperscaler deploys a Vera Rubin rack, every time an enterprise qualifies Vera for their AI factory, ARM earns a royalty. The secular tailwind here is almost perfectly constructed for ARM's business model. Amazon's Graviton, Microsoft's Cobalt, Google's Axion, Apple's silicon stack, and Qualcomm's data center push all run on ARM. And now Nvidia's Vera, which is projected to displace Intel as the largest server CPU supplier by revenue in a single fiscal year, is ARM. ARM's royalty rate on high end server chips is estimated at roughly 1 to 2% of chip selling price. At $5,000 per Vera CPU and 4 million units projected for FY2027, that is a royalty line growing from near zero to potentially $400 million to $800 million annually from Nvidia's data center CPU business alone before counting Amazon, Microsoft, Google, Apple, and Qualcomm. The total ARM addressable royalty base across all the silicon it already licenses is compounding at a rate that the current $130 billion market cap does not fully reflect. Jensen's CPU thesis is the most underappreciated catalyst in ARM's fundamental story, and the royalty compounding has barely started. Come join Milk Road Pro and get our full ARM royalty model and our entire AI trade thesis. Link below!
3:41

Jensen Huang just identified the next $200 billion market (Save this). The shift starts with a observation about agentic AI that changes everything about infrastructure. In the era of training and inference, the GPU was everything while CPU was a traffic cop, scheduling work, managing memory, dispatching tasks while the GPU did the heavy lifting. Agentic AI breaks that model entirely. An AI agent does not just run a single inference pass but rather it plans, calls tools, executes code in sandboxes, retrieves data from multiple sources and loops through complex multi-step reasoning sequences often thousands of times per second at scale. Every one of those operations runs through the CPU and the GPU sits idle waiting for the CPU to prepare the next task, supply the right context and execute the retrieval and tool calling logic fast enough to keep the accelerators fed. The CPU is now the conductor and the GPU is the orchestra and the bottleneck is the conductor falling behind. This is showing up in production AI factory utilization right now, which is exactly why Jensen built Vera from scratch rather than licensing x86. Vera achieves 40% lower peak memory latency than x86, 50% faster core to core communication, and 1.8 times the agentic sandbox performance of current x86 processors on a purpose-built architecture designed around the agentic loop. Now here is where the investment thesis gets interesting. The obvious beneficiary is Nvidia itself, and that thesis is real. Nvidia's CFO has guided for nearly $20 billion in Vera CPU revenue this fiscal year alone, a market Nvidia had zero presence in just three years ago. Intel held 60% of server CPU market share as recently as Q4 2025 and that transition is now happening at a pace Intel structurally cannot respond to. But the deeper question is, what architecture is Vera actually built on? Vera's Olympus cores are ARM compatible and every single Vera CPU deployed in every Vera Rubin rack in every data center in the world runs on ARM architecture. And ARM Holdings collects a royalty on every one of them. ARM does not make chips but rather licenses the instruction set architecture and CPU core designs that others build on top of. Every time Nvidia ships a Vera CPU, every time a hyperscaler deploys a Vera Rubin rack, every time an enterprise qualifies Vera for their AI factory, ARM earns a royalty. The secular tailwind here is almost perfectly constructed for ARM's business model. Amazon's Graviton, Microsoft's Cobalt, Google's Axion, Apple's silicon stack, and Qualcomm's data center push all run on ARM. And now Nvidia's Vera, which is projected to displace Intel as the largest server CPU supplier by revenue in a single fiscal year, is ARM. ARM's royalty rate on high end server chips is estimated at roughly 1 to 2% of chip selling price. At $5,000 per Vera CPU and 4 million units projected for FY2027, that is a royalty line growing from near zero to potentially $400 million to $800 million annually from Nvidia's data center CPU business alone before counting Amazon, Microsoft, Google, Apple, and Qualcomm. The total ARM addressable royalty base across all the silicon it already licenses is compounding at a rate that the current $130 billion market cap does not fully reflect. Jensen's CPU thesis is the most underappreciated catalyst in ARM's fundamental story, and the royalty compounding has barely started. Come join Milk Road Pro and get our full ARM royalty model and our entire AI trade thesis. Link below!

Milk Road AI

11,819 次观看 • 4 个月前

A guy who tested every AI Halo mini PC on the market picked a winner: The Minisforum MS-S1 Max They all run the same AMD Ryzen AI Max+ 395 with 128GB unified memory, so the decision came down to everything around the chip What won it: dual 10-gig Ethernet, 80 Gbit/s USB4 V2, internal power supply, a low-profile PCIe slot the competition skipped The overlooked detail: the CPU/GPU memory split quietly changes CPU performance by 15% on identical silicon The honest comparison: NVIDIA's GB10 boxes have nearly closed the price gap and still lead on software maturity. AMD's edge is Windows, gaming, and expansion in one machine The takeaway is not which box is fastest It is that local AI hardware is now a commodity, and the real differences are the boring ones: ports, service, and software
17:57

A guy who tested every AI Halo mini PC on the market picked a winner: The Minisforum MS-S1 Max They all run the same AMD Ryzen AI Max+ 395 with 128GB unified memory, so the decision came down to everything around the chip What won it: dual 10-gig Ethernet, 80 Gbit/s USB4 V2, internal power supply, a low-profile PCIe slot the competition skipped The overlooked detail: the CPU/GPU memory split quietly changes CPU performance by 15% on identical silicon The honest comparison: NVIDIA's GB10 boxes have nearly closed the price gap and still lead on software maturity. AMD's edge is Windows, gaming, and expansion in one machine The takeaway is not which box is fastest It is that local AI hardware is now a commodity, and the real differences are the boring ones: ports, service, and software

plutos

97,428 次观看 • 3 个月前

We used to buy more VRAM. Now we’re finding ways around the VRAM limit. This little DDR5 SO-DIMM setup is paired with an old NVIDIA Tesla P40 in the video. The P40 is an old datacenter GPU with 24GB of VRAM. By today’s standards, it’s not fast. But combine that with a machine carrying 128GB of system memory, and suddenly you have a very different local AI box. The GPU handles what it’s good at. The system memory gives the model somewhere to go when VRAM runs out. You obviously pay for that with bandwidth and inference speed. But this is the part I find interesting: Local AI is making “how much memory do I have?” almost as important as “how powerful is my GPU?” We’ve spent years chasing faster GPUs. Maybe the next wave of local AI hardware will be about finding smarter ways to use all the memory we already have. More VRAM is still better. But it’s no longer the only way to run a bigger model.
0:27

We used to buy more VRAM. Now we’re finding ways around the VRAM limit. This little DDR5 SO-DIMM setup is paired with an old NVIDIA Tesla P40 in the video. The P40 is an old datacenter GPU with 24GB of VRAM. By today’s standards, it’s not fast. But combine that with a machine carrying 128GB of system memory, and suddenly you have a very different local AI box. The GPU handles what it’s good at. The system memory gives the model somewhere to go when VRAM runs out. You obviously pay for that with bandwidth and inference speed. But this is the part I find interesting: Local AI is making “how much memory do I have?” almost as important as “how powerful is my GPU?” We’ve spent years chasing faster GPUs. Maybe the next wave of local AI hardware will be about finding smarter ways to use all the memory we already have. More VRAM is still better. But it’s no longer the only way to run a bigger model.

KyzoroX

35,554 次观看 • 1 个月前

THIS GUY WIRED GMKTEC EVO-X2 BOXES, A $40 USB-C BRICK AND AN RTX 3090 INTO A $6,800/MONTH OFFLINE AI FARM it looks like a messy rack with purple cables, mini pcs and a gpu lying on the desk. but the real point is brutal: one $1,800 gmktec evo-x2 can replace a $440/month ai stack, then the rack turns it into a local backend. the evo-x2 runs amd’s ryzen ai max+ 395 with 128gb of unified memory shared between cpu, gpu and npu. on linux, that gives up to 110gb of usable vram for local models without renting cloud compute. that is why the setup gets weird. qwen3-coder 30b, llama 3.3 70b, deepseek v3 and 200b-class models can run from boxes smaller than consoles. the rtx 3090 becomes a 24gb side node, not the whole system. the $40 anker nano ii 65w brick is the funny part. cheap power, tiny machines, one old gpu and local inference. the whole stack starts looking less like a workstation and more like a private ai grid. the subscription math is ugly. claude code max is $200/month, chatgpt pro is $200/month, cursor is $20/month and gemini adds another $20. that is $5,280/year before api usage even starts. point claude code at localhost, pull models through ollama and run agents all day: code reviews, scraping, transcription, research reports and client automations. 3 clients at $2,200/month turns this into a $6,800/month backend. bookmark this and read the article below
0:47

THIS GUY WIRED GMKTEC EVO-X2 BOXES, A $40 USB-C BRICK AND AN RTX 3090 INTO A $6,800/MONTH OFFLINE AI FARM it looks like a messy rack with purple cables, mini pcs and a gpu lying on the desk. but the real point is brutal: one $1,800 gmktec evo-x2 can replace a $440/month ai stack, then the rack turns it into a local backend. the evo-x2 runs amd’s ryzen ai max+ 395 with 128gb of unified memory shared between cpu, gpu and npu. on linux, that gives up to 110gb of usable vram for local models without renting cloud compute. that is why the setup gets weird. qwen3-coder 30b, llama 3.3 70b, deepseek v3 and 200b-class models can run from boxes smaller than consoles. the rtx 3090 becomes a 24gb side node, not the whole system. the $40 anker nano ii 65w brick is the funny part. cheap power, tiny machines, one old gpu and local inference. the whole stack starts looking less like a workstation and more like a private ai grid. the subscription math is ugly. claude code max is $200/month, chatgpt pro is $200/month, cursor is $20/month and gemini adds another $20. that is $5,280/year before api usage even starts. point claude code at localhost, pull models through ollama and run agents all day: code reviews, scraping, transcription, research reports and client automations. 3 clients at $2,200/month turns this into a $6,800/month backend. bookmark this and read the article below

Gipp 🦅

20,031 次观看 • 3 个月前

HERMES AGENT NOW RUNS ON AN 8GB LAPTOP GPU JUST AS EASILY AS IT RUNS ON A 128GB MINI PC Nous Research shipped the official Hermes Agent Desktop App this week. Someone pointed it at a local llama server running on an RTX 4060 with 16GB system RAM. The integration took two minutes The model behind it: Gemma 4 26B MoE, QAT quantized, running on 8GB of VRAM. A 60k token prompt held a stable 20 tokens a second, flat, no slowdown as context grew. The flags were nothing exotic, just -cmoe -c 248000 on llama.cpp What that 8GB setup does out of the box: reads and patches its own code, runs it in a terminal, debugs errors, manages GitHub repos, spawns sub-agents for parallel work. Browses the web with vision to debug a UI. Schedules cron jobs in plain language. Connects to Notion, Google Workspace, Linear, and Obsidian to manage tasks on its own That's the same agent layer running on a Minisforum MS-S1 MAX with 128GB of unified memory, 96GB of it to the GPU, holding a 120B model at 56 tokens a second instead of a 26B model at 20. Same software, same tool execution, same zero API key. The only thing that changes between an $800 laptop and a $2,000 mini PC is how big a model you can afford to run underneath it The barrier to running a real autonomous agent locally didn't just drop. It dropped all the way down to hardware most people already own
0:20

HERMES AGENT NOW RUNS ON AN 8GB LAPTOP GPU JUST AS EASILY AS IT RUNS ON A 128GB MINI PC Nous Research shipped the official Hermes Agent Desktop App this week. Someone pointed it at a local llama server running on an RTX 4060 with 16GB system RAM. The integration took two minutes The model behind it: Gemma 4 26B MoE, QAT quantized, running on 8GB of VRAM. A 60k token prompt held a stable 20 tokens a second, flat, no slowdown as context grew. The flags were nothing exotic, just -cmoe -c 248000 on llama.cpp What that 8GB setup does out of the box: reads and patches its own code, runs it in a terminal, debugs errors, manages GitHub repos, spawns sub-agents for parallel work. Browses the web with vision to debug a UI. Schedules cron jobs in plain language. Connects to Notion, Google Workspace, Linear, and Obsidian to manage tasks on its own That's the same agent layer running on a Minisforum MS-S1 MAX with 128GB of unified memory, 96GB of it to the GPU, holding a 120B model at 56 tokens a second instead of a 26B model at 20. Same software, same tool execution, same zero API key. The only thing that changes between an $800 laptop and a $2,000 mini PC is how big a model you can afford to run underneath it The barrier to running a real autonomous agent locally didn't just drop. It dropped all the way down to hardware most people already own

NO1ennn

40,079 次观看 • 3 个月前

AMD CEO LISA SU HELD A MINI PC ON STAGE THAT RUNS A 235B MODEL AND REPLACES YOUR $440/MONTH AI STACK amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on one piece of silicon. cpu and gpu share 128gb of unified memory, no separate graphics card needed the gmktec evo-x2 runs qwen3 235b fully, deepseek v3 comfortably and llama 3.3 70b with headroom. on linux you get 110gb of usable vram out of 128gb amd claimed the chip beat an nvidia rtx 5080 by more than 3x on deepseek r1 inference. a lunchbox sized pc outrunning a $1,000 discrete gpu on a real ai workload a heavy ai user pays $200 for claude code max, $200 for chatgpt pro, $20 for cursor and $20 for gemini. that's $5,280 a year and the box pays itself off in 9 to 10 months install ollama, pull the model, point claude code at localhost. same interface, nothing leaves the machine, nothing costs per request bookmark this and read the article below
0:59

AMD CEO LISA SU HELD A MINI PC ON STAGE THAT RUNS A 235B MODEL AND REPLACES YOUR $440/MONTH AI STACK amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on one piece of silicon. cpu and gpu share 128gb of unified memory, no separate graphics card needed the gmktec evo-x2 runs qwen3 235b fully, deepseek v3 comfortably and llama 3.3 70b with headroom. on linux you get 110gb of usable vram out of 128gb amd claimed the chip beat an nvidia rtx 5080 by more than 3x on deepseek r1 inference. a lunchbox sized pc outrunning a $1,000 discrete gpu on a real ai workload a heavy ai user pays $200 for claude code max, $200 for chatgpt pro, $20 for cursor and $20 for gemini. that's $5,280 a year and the box pays itself off in 9 to 10 months install ollama, pull the model, point claude code at localhost. same interface, nothing leaves the machine, nothing costs per request bookmark this and read the article below

starmex

981,873 次观看 • 3 个月前

AMD JUST KILLED THE $200 - A - MONTH AI SUBSCRIPTION - A 65-WATT BOX WITH 128GB OF UNIFIED MEMORY THAT RUNS 70B MODELS LOCALLY. that clip is amd's ryzen ai max+ 395 mini box, held up on stage. the whole pitch is one spec: 128gb of high-speed unified memory. why it matters: cpu, gpu and npu share the same 128gb pool, so a model that used to need a rack of vram just fits. what it actually runs: llama 70b in full precision on the built-in gpu open models up to ~100b parameters, locally, no cloud all of it in a mini pc pulling around 65 watts that's the part that ends the subscription: your prompts, your data, your model, on a box on your desk, not metered by the token. honest line: this runs open models, not gpt-5 or claude opus. for chat, coding help and agents on your own data it's more than enough. for the absolute frontier, the cloud still wins. and it isn't free: the box runs ~$1,500 and up. but that's a few months of a $200 sub, then it's yours forever. no monthly ai bill, no tokens metered, no prompt leaving the room. the uncomfortable part isn't that amd made a fast box. it's that "you need the cloud for ai" was a pricing choice, and unified memory just made it optional. follow RetroChainer for the hardware that turns rent into ownership, and bookmark this before your next ai renewal.
0:41

AMD JUST KILLED THE $200 - A - MONTH AI SUBSCRIPTION - A 65-WATT BOX WITH 128GB OF UNIFIED MEMORY THAT RUNS 70B MODELS LOCALLY. that clip is amd's ryzen ai max+ 395 mini box, held up on stage. the whole pitch is one spec: 128gb of high-speed unified memory. why it matters: cpu, gpu and npu share the same 128gb pool, so a model that used to need a rack of vram just fits. what it actually runs: llama 70b in full precision on the built-in gpu open models up to ~100b parameters, locally, no cloud all of it in a mini pc pulling around 65 watts that's the part that ends the subscription: your prompts, your data, your model, on a box on your desk, not metered by the token. honest line: this runs open models, not gpt-5 or claude opus. for chat, coding help and agents on your own data it's more than enough. for the absolute frontier, the cloud still wins. and it isn't free: the box runs ~$1,500 and up. but that's a few months of a $200 sub, then it's yours forever. no monthly ai bill, no tokens metered, no prompt leaving the room. the uncomfortable part isn't that amd made a fast box. it's that "you need the cloud for ai" was a pricing choice, and unified memory just made it optional. follow RetroChainer for the hardware that turns rent into ownership, and bookmark this before your next ai renewal.

RetroChainer

16,995 次观看 • 1 个月前