Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

AN AWS ENGINEER QUIETLY BUILT A 2 PETABYTE HOME SERVER FOR $9/MONTH THAT KILLS A $3,400/MONTH CLOUD STORAGE BILL the lenovo thinkstation pgx ships nvidia's gb10 grace blackwell superchip and 128gb of unified memory in a box the size of a mac mini at 1.2kg it runs an 80b...

starmexxx's profile picture

starmex

8,925 subscribers

193,226 Aufrufe • vor 2 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇
0:19

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇

ZEUS⚡️

85,803 Aufrufe • vor 3 Monaten

JENSEN HUANG UNVEILED A BOARD THAT RUNS 1 TRILLION PARAMETER AI MODELS. THE $249 NVIDIA BOX UNDER YOUR DESK KILLS A $200/MONTH AI BILL FOR $5 IN ELECTRICITY jensen held it up on stage with one hand and called it the architecture that runs the future of ai. that same technology now ships in a $249 box smaller than your wallet the jetson orin nano super pulls 7-25 watts and does 67 trillion ai operations per second. llama 3, mistral and deepseek run locally with no api fees and no data leaving your machine most developers pay $2,400 a year across chatgpt, openai api, claude pro and cursor. the jetson costs $314 in year one and $60 a year after. 2 year savings hit $4,431 install ollama with one command, change one line of code to point at localhost, and every tool built for openai works identically. zero rewrites, zero rate limits cloud subscriptions keep getting more expensive and rate limits keep getting tighter. the people who own the box in 2026 are going to look very far ahead in 2028 bookmark this and read the article below
0:15

JENSEN HUANG UNVEILED A BOARD THAT RUNS 1 TRILLION PARAMETER AI MODELS. THE $249 NVIDIA BOX UNDER YOUR DESK KILLS A $200/MONTH AI BILL FOR $5 IN ELECTRICITY jensen held it up on stage with one hand and called it the architecture that runs the future of ai. that same technology now ships in a $249 box smaller than your wallet the jetson orin nano super pulls 7-25 watts and does 67 trillion ai operations per second. llama 3, mistral and deepseek run locally with no api fees and no data leaving your machine most developers pay $2,400 a year across chatgpt, openai api, claude pro and cursor. the jetson costs $314 in year one and $60 a year after. 2 year savings hit $4,431 install ollama with one command, change one line of code to point at localhost, and every tool built for openai works identically. zero rewrites, zero rate limits cloud subscriptions keep getting more expensive and rate limits keep getting tighter. the people who own the box in 2026 are going to look very far ahead in 2028 bookmark this and read the article below

starmex

54,448 Aufrufe • vor 2 Monaten

This guy built a mini AI farm out of 4 Nvidia boxes It does not look like a data center. It looks like a stack of small machines sitting next to a laptop. But each box is a DGX Spark with Grace Blackwell inside, 128GB unified memory, and enough room to run models normal gaming GPUs cannot even open. Using the launch price from the article, 4 of them is almost $12,000 of local AI compute on one desk. That sounds expensive until you compare it to cloud GPUs. A serious AI builder can burn $1,500 to $3,000 a month renting A100s and H100s for client work, fine-tunes, agents and 70B models. He basically moved that bill from the cloud into hardware he owns. 4 Nvidia boxes. 512GB unified memory. No hourly meter running in the background. No rented GPUs eating the margin every time an agent runs too long. The funny part is most people still think local AI means a slow laptop running a toy model. Meanwhile guys like this are stacking compute at home. Save this, local AI is turning into the new mining farm.
0:11

This guy built a mini AI farm out of 4 Nvidia boxes It does not look like a data center. It looks like a stack of small machines sitting next to a laptop. But each box is a DGX Spark with Grace Blackwell inside, 128GB unified memory, and enough room to run models normal gaming GPUs cannot even open. Using the launch price from the article, 4 of them is almost $12,000 of local AI compute on one desk. That sounds expensive until you compare it to cloud GPUs. A serious AI builder can burn $1,500 to $3,000 a month renting A100s and H100s for client work, fine-tunes, agents and 70B models. He basically moved that bill from the cloud into hardware he owns. 4 Nvidia boxes. 512GB unified memory. No hourly meter running in the background. No rented GPUs eating the margin every time an agent runs too long. The funny part is most people still think local AI means a slow laptop running a toy model. Meanwhile guys like this are stacking compute at home. Save this, local AI is turning into the new mining farm.

Gipp 🦅

591,167 Aufrufe • vor 3 Monaten

MARCUS CHEN STACKED 30 MAC MINIS INTO AN AI SERVER FARM. ONE $599 MAC MINI REPLACES YOUR $200/MONTH CLAUDE CODE BILL WITH $3 IN ELECTRICITY two months ago a developer posted his claude code bill on reddit. $170 in 10 days. someone replied "i bought a mac mini m4. haven't paid anthropic since." apple stores ran out of mac minis the same week the m4 chip has 120 gb/s memory bandwidth and unified memory architecture. cpu and gpu share one pool so the model loads once and both read from it. a $599 mac mini runs ai faster than a $1,500 windows pc with a discrete gpu since january 2026 ollama supports the anthropic messages api format. claude code connects directly to your local mac mini with one environment variable. same interface, zero api costs, $0 per request a heavy developer pays $459 a month across claude code max, chatgpt pro, gemini, cursor and copilot. that's $5,508 a year. the mac mini pays off in 3 months and runs on $3 in electricity after that uber rolled out claude code to 5,000 engineers and burned through their $3.4 billion 2026 ai budget in 4 months. the people who own the hardware in 2026 are going to look very far ahead in 2028 bookmark this and read the article below
0:16

MARCUS CHEN STACKED 30 MAC MINIS INTO AN AI SERVER FARM. ONE $599 MAC MINI REPLACES YOUR $200/MONTH CLAUDE CODE BILL WITH $3 IN ELECTRICITY two months ago a developer posted his claude code bill on reddit. $170 in 10 days. someone replied "i bought a mac mini m4. haven't paid anthropic since." apple stores ran out of mac minis the same week the m4 chip has 120 gb/s memory bandwidth and unified memory architecture. cpu and gpu share one pool so the model loads once and both read from it. a $599 mac mini runs ai faster than a $1,500 windows pc with a discrete gpu since january 2026 ollama supports the anthropic messages api format. claude code connects directly to your local mac mini with one environment variable. same interface, zero api costs, $0 per request a heavy developer pays $459 a month across claude code max, chatgpt pro, gemini, cursor and copilot. that's $5,508 a year. the mac mini pays off in 3 months and runs on $3 in electricity after that uber rolled out claude code to 5,000 engineers and burned through their $3.4 billion 2026 ai budget in 4 months. the people who own the hardware in 2026 are going to look very far ahead in 2028 bookmark this and read the article below

starmex

357,679 Aufrufe • vor 2 Monaten

GEMMA 4 26B ON AN RTX 4060 WITH A 248K TOKEN CONTEXT WINDOW 20 tokens per second and a context window so large you can feed it entire codebases, books and research papers in a single prompt this is not a cloud api and not a server rack, this is a regular consumer gpu running locally with llama.cpp and q4_k_xl quantization 248k context on an 8gb vram card was not supposed to be possible and here it is just running on someone’s desk the article below covers exactly which tools and configs make this kind of setup work in 2026 ↓
0:19

GEMMA 4 26B ON AN RTX 4060 WITH A 248K TOKEN CONTEXT WINDOW 20 tokens per second and a context window so large you can feed it entire codebases, books and research papers in a single prompt this is not a cloud api and not a server rack, this is a regular consumer gpu running locally with llama.cpp and q4_k_xl quantization 248k context on an 8gb vram card was not supposed to be possible and here it is just running on someone’s desk the article below covers exactly which tools and configs make this kind of setup work in 2026 ↓

leopardracer

56,772 Aufrufe • vor 2 Monaten

i spent $26,600 on cloud GPU rentals over 14 months before i found a NVIDIA DGX Spark at $2,999 (founder's edition) or $3,999 (shipping price) it paid for itself in 6 weeks i run 200B parameter models locally now and my old cloud provider keeps sending me loyalty discount emails the math on that $26,600 is embarrassing to type out loud $1,900/month for 14 months, H100 instances on a specialist cloud provider, because anything bigger than a 70B model simply would not fit anywhere else i paid the invoices like they were a utility bill and told myself it was just the cost of doing serious AI work it took me over a year to find out it wasn't 14 months, broken down: → months 1-4: $1,400-1,600/month - felt like manageable infrastructure overhead → months 5-9: crept to $1,900-2,100 as i started running DeepSeek-class experiments, costs tracking directly with model size → months 10-12: one agent loop ran for 36 hours against a 130B model while i slept, that month hit $2,400 → month 13: ran the cumulative total for the first time, saw $23,800, felt physically sick → month 14: another $2,800 month while i waited for the hardware to ship the box is the NVIDIA DGX Spark - roughly the footprint of a large mac mini, powered by a GB10 Grace Blackwell chip with 128GB of unified LPDDR5X memory that unified memory is the whole thing an RTX 4090 has 24GB of VRAM, which means a 70B model in full BF16 precision physically does not fit, you're quantizing down or you're renting cloud, those are your options this box loads a 200B parameter model quantized and serves it through vLLM over localhost, same API interface the cloud endpoint used the migration took one line of code - i changed the base URL from the provider's endpoint to 127.0.0.1:8000 and everything just worked electricity to run continuous 200B inference locally comes out to about $12/month the payback arithmetic is almost too clean: $2,999 hardware cost against $1,900/month saved, the box paid for itself before i'd owned it two months what i didn't account for was how completely the cost model changes your behavior when there's no hourly meter running, you greenlight experiments you'd never approve on cloud - agent loops that churn for hours, running 10,000 documents through a reasoning pass at 3am, speculative fine-tuning jobs you'd normally skip because the cost felt unjustifiable i ran more experiments in the first 30 days after the box arrived than in the four months before it the loyalty discount email landed about 8 weeks after i cancelled the cloud subscription 15% off my next three months, valued customer, we'd love to have you back i didn't reply the box was already running
0:15

i spent $26,600 on cloud GPU rentals over 14 months before i found a NVIDIA DGX Spark at $2,999 (founder's edition) or $3,999 (shipping price) it paid for itself in 6 weeks i run 200B parameter models locally now and my old cloud provider keeps sending me loyalty discount emails the math on that $26,600 is embarrassing to type out loud $1,900/month for 14 months, H100 instances on a specialist cloud provider, because anything bigger than a 70B model simply would not fit anywhere else i paid the invoices like they were a utility bill and told myself it was just the cost of doing serious AI work it took me over a year to find out it wasn't 14 months, broken down: → months 1-4: $1,400-1,600/month - felt like manageable infrastructure overhead → months 5-9: crept to $1,900-2,100 as i started running DeepSeek-class experiments, costs tracking directly with model size → months 10-12: one agent loop ran for 36 hours against a 130B model while i slept, that month hit $2,400 → month 13: ran the cumulative total for the first time, saw $23,800, felt physically sick → month 14: another $2,800 month while i waited for the hardware to ship the box is the NVIDIA DGX Spark - roughly the footprint of a large mac mini, powered by a GB10 Grace Blackwell chip with 128GB of unified LPDDR5X memory that unified memory is the whole thing an RTX 4090 has 24GB of VRAM, which means a 70B model in full BF16 precision physically does not fit, you're quantizing down or you're renting cloud, those are your options this box loads a 200B parameter model quantized and serves it through vLLM over localhost, same API interface the cloud endpoint used the migration took one line of code - i changed the base URL from the provider's endpoint to 127.0.0.1:8000 and everything just worked electricity to run continuous 200B inference locally comes out to about $12/month the payback arithmetic is almost too clean: $2,999 hardware cost against $1,900/month saved, the box paid for itself before i'd owned it two months what i didn't account for was how completely the cost model changes your behavior when there's no hourly meter running, you greenlight experiments you'd never approve on cloud - agent loops that churn for hours, running 10,000 documents through a reasoning pass at 3am, speculative fine-tuning jobs you'd normally skip because the cost felt unjustifiable i ran more experiments in the first 30 days after the box arrived than in the four months before it the loyalty discount email landed about 8 weeks after i cancelled the cloud subscription 15% off my next three months, valued customer, we'd love to have you back i didn't reply the box was already running

Argona

22,355 Aufrufe • vor 2 Monaten

THIS SHELF OF MAC MINIS REPLACES $4,080 A YEAR IN AI SUBSCRIPTIONS 00:02 the camera pans across a shelf of stacked Mac minis and the trick is obvious: that silent little farm runs the models you rent every month most people pay 7 companies for AI and use 3 of the tools. they forget the rest on the credit card and call it a stack the Mac mini M4 ends that. one shared memory pool means a $599 box runs 7B and 8B models faster than Windows machines that cost twice as much ollama pull, one command. open webui in one docker line. point Claude Code at localhost and it just works it draws 10 to 30 watts, sits silent next to a router, and runs 24/7 for $3 a month in power it pays back a $20 ChatGPT Plus sub in 3 months, then saves you $4,000 a year while the frontier still rents you compute every month you wait is another $340 gone for compute that fits on a shelf
0:10

THIS SHELF OF MAC MINIS REPLACES $4,080 A YEAR IN AI SUBSCRIPTIONS 00:02 the camera pans across a shelf of stacked Mac minis and the trick is obvious: that silent little farm runs the models you rent every month most people pay 7 companies for AI and use 3 of the tools. they forget the rest on the credit card and call it a stack the Mac mini M4 ends that. one shared memory pool means a $599 box runs 7B and 8B models faster than Windows machines that cost twice as much ollama pull, one command. open webui in one docker line. point Claude Code at localhost and it just works it draws 10 to 30 watts, sits silent next to a router, and runs 24/7 for $3 a month in power it pays back a $20 ChatGPT Plus sub in 3 months, then saves you $4,000 a year while the frontier still rents you compute every month you wait is another $340 gone for compute that fits on a shelf

Fokki

12,933 Aufrufe • vor 2 Monaten

THIS GUY BUILT A MINI AI FARM OUT OF 4 NVIDIA BOXES. IT DOESN'T LOOK LIKE A DATA CENTER. IT LOOKS LIKE A STACK OF SMALL MACHINES SITTING NEXT TO A LAPTOP. BUT EACH BOX IS A DGX SPARK WITH GRACE BLACKWELL INSIDE. 4 BOXES. 512GB OF UNIFIED MEMORY. NEARLY $12,000 OF LOCAL AI COMPUTE SITTING ON ONE DESK. NO CLOUD GPU RENTAL FEES. NO HOURLY METER. NO RENTED COMPUTE EATING YOUR MARGINS. MOST PEOPLE STILL THINK LOCAL AI MEANS A SLOW LAPTOP RUNNING A TOY MODEL. MEANWHILE, PEOPLE ARE STACKING SERIOUS AI COMPUTE AT HOME. LOCAL AI IS STARTING TO LOOK LIKE THE NEW MINING FARM.
0:11

THIS GUY BUILT A MINI AI FARM OUT OF 4 NVIDIA BOXES. IT DOESN'T LOOK LIKE A DATA CENTER. IT LOOKS LIKE A STACK OF SMALL MACHINES SITTING NEXT TO A LAPTOP. BUT EACH BOX IS A DGX SPARK WITH GRACE BLACKWELL INSIDE. 4 BOXES. 512GB OF UNIFIED MEMORY. NEARLY $12,000 OF LOCAL AI COMPUTE SITTING ON ONE DESK. NO CLOUD GPU RENTAL FEES. NO HOURLY METER. NO RENTED COMPUTE EATING YOUR MARGINS. MOST PEOPLE STILL THINK LOCAL AI MEANS A SLOW LAPTOP RUNNING A TOY MODEL. MEANWHILE, PEOPLE ARE STACKING SERIOUS AI COMPUTE AT HOME. LOCAL AI IS STARTING TO LOOK LIKE THE NEW MINING FARM.

Vivek Sen

31,812 Aufrufe • vor 17 Tagen

A HARDWARE MODDER BOLTED A BATTERY PACK ONTO A $599 MAC MINI M4 AND TURNED A DESK SERVER INTO A BACKPACK AI RIG THAT RUNS FOR EIGHT HOURS WITHOUT A WALL OUTLET he posts a video of the mac mini with a slim battery module clamped to the side, USB-C plugged into the back. no power brick, no outlet, just the 10 to 30W draw of the M4 chip pulling from a 20,000mAh bank. a full work day on one charge. this is not a portable laptop trick. it is a desk-class AI server that fits in a sling bag the same $599 mac mini i recommended in last week's article as the silent default, now untethered. pair it with a mobile hotspot or a Starlink mini and you have a fully off-grid LLM stack that answers questions in a forest, on a plane, in a power outage. the model lives in the box, the battery powers the box, that is the whole stack this is not a gadget. it is the first time the words "AI server" and "fits in your bag" belong in the same sentence
0:19

A HARDWARE MODDER BOLTED A BATTERY PACK ONTO A $599 MAC MINI M4 AND TURNED A DESK SERVER INTO A BACKPACK AI RIG THAT RUNS FOR EIGHT HOURS WITHOUT A WALL OUTLET he posts a video of the mac mini with a slim battery module clamped to the side, USB-C plugged into the back. no power brick, no outlet, just the 10 to 30W draw of the M4 chip pulling from a 20,000mAh bank. a full work day on one charge. this is not a portable laptop trick. it is a desk-class AI server that fits in a sling bag the same $599 mac mini i recommended in last week's article as the silent default, now untethered. pair it with a mobile hotspot or a Starlink mini and you have a fully off-grid LLM stack that answers questions in a forest, on a plane, in a power outage. the model lives in the box, the battery powers the box, that is the whole stack this is not a gadget. it is the first time the words "AI server" and "fits in your bag" belong in the same sentence

shmidt

31,619 Aufrufe • vor 2 Monaten

KIMI K2.6 SERVERS BURN 30 MILLION LITERS OF WATER A MONTH. INDIE DEVS USE THE SAME MODEL FOR $30 IN TOKENS TO LAUNCH $20,000/MONTH APPS IN A WEEKEND kimi k2.6 sits at number 1 on the openrouter leaderboard processing 1.58 trillion tokens a week. more than claude sonnet 4.6 and deepseek combined indie developers who launched in 2024-2025 are making $10,000-20,000 a month solo. no team, no office, $20 in starting costs. most aren't even senior devs the stack is next.js, supabase, stripe and kimi k2.6. you give the model 5 open source repos as reference and it assembles the product from the best parts of each 3,000 paying users at $9.99 a month is $29,970 in revenue. infrastructure costs $1,235. net profit lands at $28,735 with a 95% margin most people will bookmark this and forget. the ones who ship this weekend get a 6-12 month head start in app store rankings over everyone who starts later bookmark this and read the article below
0:10

KIMI K2.6 SERVERS BURN 30 MILLION LITERS OF WATER A MONTH. INDIE DEVS USE THE SAME MODEL FOR $30 IN TOKENS TO LAUNCH $20,000/MONTH APPS IN A WEEKEND kimi k2.6 sits at number 1 on the openrouter leaderboard processing 1.58 trillion tokens a week. more than claude sonnet 4.6 and deepseek combined indie developers who launched in 2024-2025 are making $10,000-20,000 a month solo. no team, no office, $20 in starting costs. most aren't even senior devs the stack is next.js, supabase, stripe and kimi k2.6. you give the model 5 open source repos as reference and it assembles the product from the best parts of each 3,000 paying users at $9.99 a month is $29,970 in revenue. infrastructure costs $1,235. net profit lands at $28,735 with a 95% margin most people will bookmark this and forget. the ones who ship this weekend get a 6-12 month head start in app store rankings over everyone who starts later bookmark this and read the article below

starmex

31,826 Aufrufe • vor 2 Monaten

Everyone assumes serious AI compute means renting someone else's datacenter This is the datacenter 1,000 Mac Mini M4s on steel shelves $599 each, paid once Replacing a $14K a month cloud GPU bill It started much smaller A dev posted his Claude Code bill, $170 in 10 days The top reply got bigger than the post "I bought a Mac Mini M4, haven't paid Anthropic since" 10-30 watts a box instead of 300-500 120 GB/s unified memory, the model loads once Ollama now speaks the Anthropic Messages API So Claude Code can point at the box under your desk instead of Anthropic's servers Nobody claims these beat frontier models They claim most of your prompts never needed one The bill was never the price of intelligence It's the price of renting it for work that never needed renting Full breakdown below
0:13

Everyone assumes serious AI compute means renting someone else's datacenter This is the datacenter 1,000 Mac Mini M4s on steel shelves $599 each, paid once Replacing a $14K a month cloud GPU bill It started much smaller A dev posted his Claude Code bill, $170 in 10 days The top reply got bigger than the post "I bought a Mac Mini M4, haven't paid Anthropic since" 10-30 watts a box instead of 300-500 120 GB/s unified memory, the model loads once Ollama now speaks the Anthropic Messages API So Claude Code can point at the box under your desk instead of Anthropic's servers Nobody claims these beat frontier models They claim most of your prompts never needed one The bill was never the price of intelligence It's the price of renting it for work that never needed renting Full breakdown below

Skaly_Bull

10,581 Aufrufe • vor 14 Tagen

HE STRAPPED A BATTERY TO A $599 MAC MINI AND TURNED A DESK COMPUTER INTO A 14-HOUR PORTABLE AI WORKSTATION 00:03 the battery slides onto the side of the mac mini and the whole setup stops behaving like a desk machine. now it can run from a backpack, power a screen, hold local files and keep working without asking for an outlet. that changes the use case completely. instead of renting another cloud box, one silent computer can handle research dumps, meeting notes, scraped pages, voice transcripts and small automation jobs from almost anywhere. with claude connected, it becomes a moving command center. 45-minute calls become summaries, 120 saved links become organized notes, and messy project folders get cleaned while the machine quietly keeps working in the background. the interesting number is not the battery size. it is the avoided rent. one portable local box can replace $25 storage, $39 automation, $20 transcription and another $30 vps bill if the workflow is built correctly. this is no longer just a desktop. it becomes a portable ai machine that keeps working long after you leave the desk. bookmark this before portable ai becomes the new normal.
0:19

HE STRAPPED A BATTERY TO A $599 MAC MINI AND TURNED A DESK COMPUTER INTO A 14-HOUR PORTABLE AI WORKSTATION 00:03 the battery slides onto the side of the mac mini and the whole setup stops behaving like a desk machine. now it can run from a backpack, power a screen, hold local files and keep working without asking for an outlet. that changes the use case completely. instead of renting another cloud box, one silent computer can handle research dumps, meeting notes, scraped pages, voice transcripts and small automation jobs from almost anywhere. with claude connected, it becomes a moving command center. 45-minute calls become summaries, 120 saved links become organized notes, and messy project folders get cleaned while the machine quietly keeps working in the background. the interesting number is not the battery size. it is the avoided rent. one portable local box can replace $25 storage, $39 automation, $20 transcription and another $30 vps bill if the workflow is built correctly. this is no longer just a desktop. it becomes a portable ai machine that keeps working long after you leave the desk. bookmark this before portable ai becomes the new normal.

Gipp 🦅

1,788,586 Aufrufe • vor 2 Monaten

forget the $699 AI pins. this $8 chip just shattered the barrier for local AI hardware. a developer just forced a 28.9 million-parameter LLM onto a standard ESP32-S3 microcontroller. it costs roughly 8 dollars, runs completely offline, and draws the power of a single LED. conventional wisdom said a model of this size simply would not fit. the chip only has 512 KB of fast SRAM and 16 MB of flash. the breakthrough is architectural. the developer moved the bulk of the embedding table into flash memory and memory-mapped it. the chip only needs to pull about 450 bytes per token, keeping the active working memory inside the fast SRAM. this means you can now embed a capable language model into a physical node for the price of two coffees. and we are already seeing the beginnings of this custom physical hardware. in the video, a creator built a minimalist voice-controlled universal remote using an ESP32. it captures voice and remotely controls the computer over bluetooth LE. he simply says "open chrome and open 20 new tabs", and the custom hardware executes it instantly. we have spent years watching model sizes explode upward. but the true frontier is the opposite direction. when an eight-dollar chip can power offline intelligence and custom physical interfaces, AI becomes local infrastructure rather than a cloud service.
0:19

forget the $699 AI pins. this $8 chip just shattered the barrier for local AI hardware. a developer just forced a 28.9 million-parameter LLM onto a standard ESP32-S3 microcontroller. it costs roughly 8 dollars, runs completely offline, and draws the power of a single LED. conventional wisdom said a model of this size simply would not fit. the chip only has 512 KB of fast SRAM and 16 MB of flash. the breakthrough is architectural. the developer moved the bulk of the embedding table into flash memory and memory-mapped it. the chip only needs to pull about 450 bytes per token, keeping the active working memory inside the fast SRAM. this means you can now embed a capable language model into a physical node for the price of two coffees. and we are already seeing the beginnings of this custom physical hardware. in the video, a creator built a minimalist voice-controlled universal remote using an ESP32. it captures voice and remotely controls the computer over bluetooth LE. he simply says "open chrome and open 20 new tabs", and the custom hardware executes it instantly. we have spent years watching model sizes explode upward. but the true frontier is the opposite direction. when an eight-dollar chip can power offline intelligence and custom physical interfaces, AI becomes local infrastructure rather than a cloud service.

ard

425,950 Aufrufe • vor 1 Monat

$200 to $20. That's how your AI costs will change if you switch from Claude Code Max to the MiniMax (official) Token Plan Plus in Kilo. MiniMax's latest model, M3, performs at 79% of Sonnet 5's level for a tenth of the price, and the plan gets you 1.7B tokens per month through Kilo. See how much you'd save with model freedom:
0:19

$200 to $20. That's how your AI costs will change if you switch from Claude Code Max to the MiniMax (official) Token Plan Plus in Kilo. MiniMax's latest model, M3, performs at 79% of Sonnet 5's level for a tenth of the price, and the plan gets you 1.7B tokens per month through Kilo. See how much you'd save with model freedom:

Kilo

25,441 Aufrufe • vor 1 Monat

THREE 3090s ON ONE BOARD GIVE YOU 72GB OF VRAM AND KILL YOUR $200 CLAUDE CODE AND $200 OPENAI BILL people are pulling three used 3090s off ebay for around $2,100 total and stacking them in one tower to build a dedicated ai rig. that pools 72gb of vram for less than what a single rtx 5090 retails for alibaba shipped qwen 3.6 27b in april under apache 2.0. on realworldqa vision it scores 84.1 against claude 4.5 opus at 77.0. on ifbench instructions it lands at 76.5 against claude's 58.0 a single 3090 already runs qwen 3.6 27b with eight gigs of headroom. three of them in parallel handle larger models like deepseek r1 70b and qwen 235b without breaking a sweat a heavy ai user pays $200 claude code, $200 chatgpt pro plus $40 cursor and gemini. that's $5,280 a year and the rig pays itself off before month nine on $8 a month in electricity setup is one shell command for ollama, one to pull the model, one environment variable to point claude code at localhost. cli stays identical, nothing leaves the network, requests stop costing money bookmark this and read the article below
0:16

THREE 3090s ON ONE BOARD GIVE YOU 72GB OF VRAM AND KILL YOUR $200 CLAUDE CODE AND $200 OPENAI BILL people are pulling three used 3090s off ebay for around $2,100 total and stacking them in one tower to build a dedicated ai rig. that pools 72gb of vram for less than what a single rtx 5090 retails for alibaba shipped qwen 3.6 27b in april under apache 2.0. on realworldqa vision it scores 84.1 against claude 4.5 opus at 77.0. on ifbench instructions it lands at 76.5 against claude's 58.0 a single 3090 already runs qwen 3.6 27b with eight gigs of headroom. three of them in parallel handle larger models like deepseek r1 70b and qwen 235b without breaking a sweat a heavy ai user pays $200 claude code, $200 chatgpt pro plus $40 cursor and gemini. that's $5,280 a year and the rig pays itself off before month nine on $8 a month in electricity setup is one shell command for ollama, one to pull the model, one environment variable to point claude code at localhost. cli stays identical, nothing leaves the network, requests stop costing money bookmark this and read the article below

starmex

16,719 Aufrufe • vor 2 Monaten

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.
0:10

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,841,161 Aufrufe • vor 4 Monaten

THAT $70 "RUN YOUR OWN LLMS" PI KIT CAN'T RUN A SINGLE LLM. IT'S A VISION CHIP WITH NO RAM. that clip sells a raspberry pi 5 in a slick case with an ai accelerator and the caption "your own llms." clean build, fun kit. the claim is where it breaks. the fine print: the popular $70 pi ai kit uses a hailo-8l, 13 tops. it's built for vision, object detection and image processing, and it has no memory of its own. so it cannot run large language models. full stop the board that actually can is a different one: the newer ai hat+ 2, hailo-10h, 40 tops, with 8gb of dedicated ram. that's $130, not $70 and even that runs only tiny models. llama 3.2 at 1b, qwen 2.5 at 1.5b, deepseek r1 at 1.5b. edge llms live in the 1-7b range, against cloud models at 500b to 2 trillion so the honest pitch: for $130 you can run a very small language model on a pi, slowly, as a fun learning project. that's real and it's cool. "your own llms" on a $70 vision kit is not. why this keeps happening: "ai kit" and a big "tops" number sell. tops sounds like intelligence. but tops measures vision-style math, not whether the chip has the memory to hold a language model. the spec that matters for llms is ram, and the cheap kit has none. the honest caveats, both ways: the $70 kit is genuinely great, just at vision. cameras, object detection, that's its job the $130 hat really does run small llms locally, which a pi couldn't do at all two years ago. that's progress "small" is the load-bearing word. don't expect gpt at home on a pi the takeaway: before you buy a kit because the caption says llm, check two numbers. not the tops. the ram, and the size of the model it can actually load. no 70-dollar miracle, no gpt in a pi case, no tops number that means what you think. save this before you buy the wrong kit for the word on the box.
0:17

THAT $70 "RUN YOUR OWN LLMS" PI KIT CAN'T RUN A SINGLE LLM. IT'S A VISION CHIP WITH NO RAM. that clip sells a raspberry pi 5 in a slick case with an ai accelerator and the caption "your own llms." clean build, fun kit. the claim is where it breaks. the fine print: the popular $70 pi ai kit uses a hailo-8l, 13 tops. it's built for vision, object detection and image processing, and it has no memory of its own. so it cannot run large language models. full stop the board that actually can is a different one: the newer ai hat+ 2, hailo-10h, 40 tops, with 8gb of dedicated ram. that's $130, not $70 and even that runs only tiny models. llama 3.2 at 1b, qwen 2.5 at 1.5b, deepseek r1 at 1.5b. edge llms live in the 1-7b range, against cloud models at 500b to 2 trillion so the honest pitch: for $130 you can run a very small language model on a pi, slowly, as a fun learning project. that's real and it's cool. "your own llms" on a $70 vision kit is not. why this keeps happening: "ai kit" and a big "tops" number sell. tops sounds like intelligence. but tops measures vision-style math, not whether the chip has the memory to hold a language model. the spec that matters for llms is ram, and the cheap kit has none. the honest caveats, both ways: the $70 kit is genuinely great, just at vision. cameras, object detection, that's its job the $130 hat really does run small llms locally, which a pi couldn't do at all two years ago. that's progress "small" is the load-bearing word. don't expect gpt at home on a pi the takeaway: before you buy a kit because the caption says llm, check two numbers. not the tops. the ram, and the size of the model it can actually load. no 70-dollar miracle, no gpt in a pi case, no tops number that means what you think. save this before you buy the wrong kit for the word on the box.

RetroChainer

11,100 Aufrufe • vor 1 Monat

I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.
0:12

I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.

Alok

170,442 Aufrufe • vor 1 Monat