Antid
@antisadh • 1,251 subscribers
studying AI daily | helping you save & make money with it
Videos
ONE BUILDER WIRED 4 RASPBERRY PI MODULES INTO A €450 AI CLUSTER WITH 64GB OF RAM AND KILLED HIS $200/MONTH CHATGPT SUBSCRIPTION 00:37 he points at the terminal, "loaded, loaded, loaded - 4 nodes synced over gigabit, 64 gigabytes total, llama 3.1 partitioned across the cluster" he stacked 4 raspberry pi CM5 lite modules with 16 gigabytes each onto a sipeed nanocluster board, the whole rig runs off a single 65 watt power supply and a gigabit internal switch distributed llama partitions the model across the nodes with synchronized workloads handled over the internal fabric, no quantization needed, the cluster hits about 30 tokens per second on small models a heavy developer pays $200 a month for chatgpt pro and another $200 for claude code, this rig cost €450 in parts and breaks even on the stack in month 2 most people will keep paying anthropic and openai forever, a few will spend a weekend wiring 4 raspberry pi modules and never see a subscription invoice again the window is open, follow and bookmark before it closes
Antid63,946 Aufrufe • vor 20 Tagen
A $40 BC250 BOARD WITH 16GB GDDR6 GETS A GITHUB FIRMWARE FLASH, THE DEFAULT 8/8 CPU/GPU SPLIT REWRITES TO 0.5/15.5 AND OPENS 15.5GB OF UNIFIED MEMORY TO OLLAMA, DEAD CRYPTO HARDWARE JUST DOUBLED ITS AI CEILING 01:08 the operator points at the chart, "you only give 512 megs to the GPU part and that's reserved, that means all the rest of the RAM is available to either the CPU or the GPU" a BC250 ships from the factory with 16GB GDDR6 split 8/8 between Oberon CPU cores and the RDNA 2 iGPU, the iGPU only ever uses 4-5GB during inference, the other 3-4GB sits stranded SEC Bolt's firmware mod on the moth-enjoyer GitHub flashes a 0.5/15.5 split, the GPU reserves 512MB and 15.5GB falls into a unified pool that ollama treats as VRAM, qwen 3.6 14B at 4 bit fits with 2GB context headroom 15.5GB on a $40 board with a $40 CH347 flasher beats the $249 Jetson Orin Nano's 8GB by nearly 2x at 1/6th the price, the firmware is open source, the dump backup process takes 4 minutes per board your map's tier zero floor was the OptiPlex at $35-50 with iGPU only, the modded BC250 sits at the same $40 mark with 15.5GB of GDDR6 and an actual RDNA 2 GPU, the buyer who flashes once unlocks a 13B class local AI host for the price of a dinner the window is open, follow and bookmark before it closes
Antid25,619 Aufrufe • vor 11 Tagen
A 24 BAY DELL POWEREDGE T550 TOWER SERVER FITS IN A HOME OFFICE WITHOUT A RACK, 384TB OF RAW SAS STORAGE HOSTS EVERY OPEN SOURCE LLM EVER RELEASED LOCALLY, THE HOME AI LIBRARY TIER UNDER YOUR MAP 00:25 the reseller spins the chassis around, "it's a 24 bay Dell T550 tower server man, look it's got a boss card for dual NVMe boot SSDs" a refurbished T550 ships at $900-1,100 with single Xeon Silver, 64GB DDR4 ECC and 24 hot swap SAS bays in a tower form factor that fits next to a desk without a server rack 24x 16TB SAS drives at $80 each on eBay equals 384TB raw storage for $1,920, the same array on enterprise Mac Studio Pro tier would cost $35,000 in thunderbolt JBOD enclosures Llama 3.3 70B is 42GB, Qwen3-235B is 110GB, DeepSeek-V3 is 100GB, Mistral Large is 80GB, the entire ollama public model library fits inside 4TB, you can mirror every open weights release for the next decade your map covers compute boxes that swap one model at a time, the T550 is the model warehouse tier that feeds them, a homelab operator wires the tower over 10GbE to a mac mini and the mac never waits 30 seconds to pull a swap the window is open, follow and bookmark before it closes
Antid17,903 Aufrufe • vor 12 Tagen
Keine weiteren Inhalte verfügbar