Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Microsoft killed the GPU mafia 🤯 They finally open-sourced their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs. - 6.17x faster inference - 82.2% less energy on CPUs 100% Open Source.

Oliver Prompts

19,244 subscribers

1,628,412 Aufrufe • vor 5 Monaten •via X (Twitter)

Gesundheit & Wellness Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

China just changed the game 🤯 They just dropped an open-sourced model that burns only 1% of the tokens compared to your favorite US models. It's called Ling-2.6-1T and it goes toe-to-toe with Claude and GPT on agentic coding. → 1 trillion params → 72.2% on SWE-bench Verified → 256K context window → Free on OpenRouter → Plugs directly into Claude Code 100% Open Source.

China just changed the game 🤯 They just dropped an open-sourced model that burns only 1% of the tokens compared to your favorite US models. It's called Ling-2.6-1T and it goes toe-to-toe with Claude and GPT on agentic coding. → 1 trillion params → 72.2% on SWE-bench Verified → 256K context window → Free on OpenRouter → Plugs directly into Claude Code 100% Open Source.

Superman

140,111 Aufrufe • vor 5 Tagen

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.

Superman

102,957 Aufrufe • vor 4 Tagen

Holy sh!t ! OpenAI will have their custom inference chips ready in just a few months and deployed at scale by the end of the year! 🤯 Training chip = The heavy lifters that require massive amounts of data and power to build and teach the AI models from scratch. Inference chip = The specialized, highly efficient chips that actually run the AI and generate the answers in real-time when you use it. This is going to help OpenAI drastically cut down their massive compute costs, speed up model reasoning times, and finally break free from relying entirely on Nvidia to scale their operations.

Holy sh!t ! OpenAI will have their custom inference chips ready in just a few months and deployed at scale by the end of the year! 🤯 Training chip = The heavy lifters that require massive amounts of data and power to build and teach the AI models from scratch. Inference chip = The specialized, highly efficient chips that actually run the AI and generate the answers in real-time when you use it. This is going to help OpenAI drastically cut down their massive compute costs, speed up model reasoning times, and finally break free from relying entirely on Nvidia to scale their operations.

Chris

60,278 Aufrufe • vor 4 Monaten

let me save you 3 hours of head scratching. if you're running local models like Qwen3.5-35B-A3B through Claude Code via llama.cpp's Anthropic endpoint, the chain will break every 3 to 5 minutes. tool call fails. flow stops. you reprompt. it recovers. 2 minutes later it stops again. the model is fine. the harness chokes on local inference latency. switch to OpenCode. same localhost endpoint. same model. same GPU. the chain doesn't break. the tradeoff: OpenCode sometimes loops. the model forgets what it already read and repeats the same tool call. but a loop you can interrupt. a broken chain kills your momentum and you start over. watch both side by side. proprietary agent vs open source agent. same 3B model. different failure modes. pick your poison.

let me save you 3 hours of head scratching. if you're running local models like Qwen3.5-35B-A3B through Claude Code via llama.cpp's Anthropic endpoint, the chain will break every 3 to 5 minutes. tool call fails. flow stops. you reprompt. it recovers. 2 minutes later it stops again. the model is fine. the harness chokes on local inference latency. switch to OpenCode. same localhost endpoint. same model. same GPU. the chain doesn't break. the tradeoff: OpenCode sometimes loops. the model forgets what it already read and repeats the same tool call. but a loop you can interrupt. a broken chain kills your momentum and you start over. watch both side by side. proprietary agent vs open source agent. same 3B model. different failure modes. pick your poison.

Sudo su

72,501 Aufrufe • vor 4 Monaten

Woow Tencent has released a model 100% open source which is better than Google Translate... And it runs locally on your phone?! - Only 440MB (!!) - 33 languages - On-device translation So we have now a local Google Translate which is running completely offline. They are using AngelSlim which their compression toolkit. That's why the model is so efficient.

Woow Tencent has released a model 100% open source which is better than Google Translate... And it runs locally on your phone?! - Only 440MB (!!) - 33 languages - On-device translation So we have now a local Google Translate which is running completely offline. They are using AngelSlim which their compression toolkit. That's why the model is so efficient.

Paul Couvert

142,071 Aufrufe • vor 2 Monaten

90% of "AI developers" just download pre packaged GGUF files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.

Alok

62,133 Aufrufe • vor 10 Tagen

From the e/acc trenches to watching Extropic ship. Congrats Beff @ NeurIPS can't wait to run this on actual stochastic silicon instead of simulating it digitally. 65k spins finding equilibrium using THRML. Extropic's THRML library built for their thermodynamic hardware. Currently Running on GPU/CPU now, but one day we might all be running quantum on our devices. Traditional computing fights physics (force bits to 0/1). Thermodynamic computing uses physics (p-bits that naturally fluctuate). For sampling problems, that's orders of magnitude more efficient. This is energy-based inference in your hands. I would have never ventured into looking into this, still way over my head but its so fucking cool wish I could have had the capital to put cheques into extropic, nous_research and rysana but lets build to the top.

From the e/acc trenches to watching Extropic ship. Congrats Beff @ NeurIPS can't wait to run this on actual stochastic silicon instead of simulating it digitally. 65k spins finding equilibrium using THRML. Extropic's THRML library built for their thermodynamic hardware. Currently Running on GPU/CPU now, but one day we might all be running quantum on our devices. Traditional computing fights physics (force bits to 0/1). Thermodynamic computing uses physics (p-bits that naturally fluctuate). For sampling problems, that's orders of magnitude more efficient. This is energy-based inference in your hands. I would have never ventured into looking into this, still way over my head but its so fucking cool wish I could have had the capital to put cheques into extropic, nous_research and rysana but lets build to the top.

Lantos (e/acc)

41,208 Aufrufe • vor 8 Monaten

Litter iOS just got a beta (TestFlight) release so now you can install it and try it out. It's a remote for codex! - Finds all the computers on your local network (or tailscale) that run codex app-server or ssh - Lets you continue any session or start a new one - Experimental local codex with oauth support Free, fully open-source, fully native, with android release coming soon! Thanks to Maky drumroll.dev ☄️ PincentΞ for the contributions! TestFlight ⬇️

Litter iOS just got a beta (TestFlight) release so now you can install it and try it out. It's a remote for codex! - Finds all the computers on your local network (or tailscale) that run codex app-server or ssh - Lets you continue any session or start a new one - Experimental local codex with oauth support Free, fully open-source, fully native, with android release coming soon! Thanks to Maky drumroll.dev ☄️ PincentΞ for the contributions! TestFlight ⬇️

SIGKITTEN

68,308 Aufrufe • vor 4 Monaten

Google Translate is cooked after this. A developer built a local AI translation engine that runs 40 languages entirely on your own laptop. It's called LibreTranslate. No API key. No usage limits. No sending your documents to Google's servers. You install it once. It runs forever. Here's what it handles: → Paste text. Translated instantly. → Drop in a file. Outputs the translated version. → Point it at a URL. Returns the page in your language. → Build it into your own app via its local REST API. The speed is not the story. The privacy is. Google Translate reads every sentence you paste into it. Legal contracts. Medical records. Internal emails. Client documents. Every word goes to their servers and stays there. LibreTranslate runs entirely offline. Nothing leaves your machine. Ever. The numbers: → 40 languages supported → Runs on CPU -- no GPU needed → Self-hosted in under 5 minutes → REST API built in for developers → 10K+ stars on GitHub 100% open source. MIT licensed. Price: $0. Google charges nothing for Translate either but it charges you something else. GitHub:

Google Translate is cooked after this. A developer built a local AI translation engine that runs 40 languages entirely on your own laptop. It's called LibreTranslate. No API key. No usage limits. No sending your documents to Google's servers. You install it once. It runs forever. Here's what it handles: → Paste text. Translated instantly. → Drop in a file. Outputs the translated version. → Point it at a URL. Returns the page in your language. → Build it into your own app via its local REST API. The speed is not the story. The privacy is. Google Translate reads every sentence you paste into it. Legal contracts. Medical records. Internal emails. Client documents. Every word goes to their servers and stays there. LibreTranslate runs entirely offline. Nothing leaves your machine. Ever. The numbers: → 40 languages supported → Runs on CPU -- no GPU needed → Self-hosted in under 5 minutes → REST API built in for developers → 10K+ stars on GitHub 100% open source. MIT licensed. Price: $0. Google charges nothing for Translate either but it charges you something else. GitHub:

Rimsha Bhardwaj

88,926 Aufrufe • vor 1 Monat

THIS GUY BOUGHT A $2,400 NVIDIA BOX AND SAVED $18,700/YEAR ON CLOUD GPUS WITHOUT RENTING SERVERS AGAIN the entire setup runs on one rule - stop paying every time you want to test something most people run 20 small AI experiments in the cloud and think it’s cheap because each one looks harmless - then the invoice comes in and suddenly their “side project” has the same monthly cost as a car payment he made the same mistake for months and it slowly killed the way he worked one box, one desk, local models - and now he can run tests overnight without thinking about hourly GPU prices $18,700/year saved by a little NVIDIA box he can literally hold in his hands

THIS GUY BOUGHT A $2,400 NVIDIA BOX AND SAVED $18,700/YEAR ON CLOUD GPUS WITHOUT RENTING SERVERS AGAIN the entire setup runs on one rule - stop paying every time you want to test something most people run 20 small AI experiments in the cloud and think it’s cheap because each one looks harmless - then the invoice comes in and suddenly their “side project” has the same monthly cost as a car payment he made the same mistake for months and it slowly killed the way he worked one box, one desk, local models - and now he can run tests overnight without thinking about hourly GPU prices $18,700/year saved by a little NVIDIA box he can literally hold in his hands

Gipp 🦅

12,484 Aufrufe • vor 1 Monat

🚨 Alibaba just open sourced a GUI agent that lives inside your webpage and controls it with natural language. It's called Page Agent and it's not a browser extension. It's pure JavaScript no Python, no Puppeteer, no headless browser, no screenshots. Just one script tag and your web app understands natural language. Here's what it actually does: → Embed it with a single tag or npm install → Control any web interface with plain English commands → Text-based DOM manipulation no OCR, no vision models needed → Bring your own LLM (GPT, Claude, Qwen, anything) → Ships a built-in UI with human-in-the-loop support → Turn 20-click ERP/CRM workflows into one sentence → Optional Chrome extension for multi-tab agent tasks → Works on any web app SaaS, admin panels, internal tools Companies are charging $30/month for AI copilots built on this exact idea. This is 3 lines of code. Your users. Your interface. The AI copilot layer for every web app just got open sourced. 1.6K stars. 100% Open Source. (Link in the comments)

🚨 Alibaba just open sourced a GUI agent that lives inside your webpage and controls it with natural language. It's called Page Agent and it's not a browser extension. It's pure JavaScript no Python, no Puppeteer, no headless browser, no screenshots. Just one script tag and your web app understands natural language. Here's what it actually does: → Embed it with a single tag or npm install → Control any web interface with plain English commands → Text-based DOM manipulation no OCR, no vision models needed → Bring your own LLM (GPT, Claude, Qwen, anything) → Ships a built-in UI with human-in-the-loop support → Turn 20-click ERP/CRM workflows into one sentence → Optional Chrome extension for multi-tab agent tasks → Works on any web app SaaS, admin panels, internal tools Companies are charging $30/month for AI copilots built on this exact idea. This is 3 lines of code. Your users. Your interface. The AI copilot layer for every web app just got open sourced. 1.6K stars. 100% Open Source. (Link in the comments)

Ihtesham Ali

135,384 Aufrufe • vor 4 Monaten

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Alok

292,096 Aufrufe • vor 1 Monat

K-Means is simple. Making it fast on GPU isn't. Flash-KMeans is an IO-aware implementation of exact k-means that rethinks the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 30x speedup over cuML - 200x speedup over FAISS Using the same exact algorithm, just engineered for today’s hardware. At the million-scale, Flash-KMeans can complete a k-means iteration in milliseconds. Here's why this matters today: K-means has always been an offline primitive. Something you run once to preprocess data and move on. These speedups change that. ↳ Vector databases like FAISS use k-means to build search indices. Faster k-means means you can re-index dynamically as data changes, not batch it overnight. ↳ LLM quantization methods need k-means to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Millisecond k-means makes it viable to run this inside the inference loop, not just in preprocessing. The 200x over FAISS is the number to internalize. FAISS is the industry standard. Most production vector search systems sit on top of it. Link to the paper and code in next tweet!

K-Means is simple. Making it fast on GPU isn't. Flash-KMeans is an IO-aware implementation of exact k-means that rethinks the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 30x speedup over cuML - 200x speedup over FAISS Using the same exact algorithm, just engineered for today’s hardware. At the million-scale, Flash-KMeans can complete a k-means iteration in milliseconds. Here's why this matters today: K-means has always been an offline primitive. Something you run once to preprocess data and move on. These speedups change that. ↳ Vector databases like FAISS use k-means to build search indices. Faster k-means means you can re-index dynamically as data changes, not batch it overnight. ↳ LLM quantization methods need k-means to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Millisecond k-means makes it viable to run this inside the inference loop, not just in preprocessing. The 200x over FAISS is the number to internalize. FAISS is the industry standard. Most production vector search systems sit on top of it. Link to the paper and code in next tweet!

Daily Dose of Data Science

23,748 Aufrufe • vor 2 Monaten

K-Means is simple. Making it fast on GPU isn't. Flash-KMeans is an IO-aware implementation of exact k-means that rethinks the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 30x speedup over cuML - 200x speedup over FAISS Using the same exact algorithm, just engineered for today’s hardware. At the million-scale, Flash-KMeans can complete a k-means iteration in milliseconds. Here's why this matters today: K-means has always been an offline primitive. Something you run once to preprocess data and move on. These speedups change that. ↳ Vector databases like FAISS use k-means to build search indices. Faster k-means means you can re-index dynamically as data changes, not batch it overnight. ↳ LLM quantization methods need k-means to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Millisecond k-means makes it viable to run this inside the inference loop, not just in preprocessing. The 200x over FAISS is the number to internalize. FAISS is the industry standard. Most production vector search systems sit on top of it. Link to the paper and code in next tweet!

K-Means is simple. Making it fast on GPU isn't. Flash-KMeans is an IO-aware implementation of exact k-means that rethinks the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 30x speedup over cuML - 200x speedup over FAISS Using the same exact algorithm, just engineered for today’s hardware. At the million-scale, Flash-KMeans can complete a k-means iteration in milliseconds. Here's why this matters today: K-means has always been an offline primitive. Something you run once to preprocess data and move on. These speedups change that. ↳ Vector databases like FAISS use k-means to build search indices. Faster k-means means you can re-index dynamically as data changes, not batch it overnight. ↳ LLM quantization methods need k-means to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Millisecond k-means makes it viable to run this inside the inference loop, not just in preprocessing. The 200x over FAISS is the number to internalize. FAISS is the industry standard. Most production vector search systems sit on top of it. Link to the paper and code in next tweet!

Akshay 🚀

36,317 Aufrufe • vor 4 Monaten

Windows 11’s June 2026 Patch Tuesday update is out today, and the biggest change is Performance, not AI. Windows 11 KB5094126 finally brings Microsoft’s new Low Latency Profile to everyone. Microsoft calls it a “General Performance” improvement, but the idea is simple: when you open Start, Search, Action Center, or key shell experiences, Windows briefly pushes the CPU to max frequency for 1–3 seconds so the UI responds faster. I tested it earlier, and the difference is most obvious on budget PCs. Start feels snappier, Search opens faster, and those tiny Windows 11 micro-stutters are reduced. The catch: because of Microsoft’s Controlled Feature Rollout, installing today’s update does not guarantee it is enabled yet. You can verify it with HWiNFO by watching CPU frequency spikes while opening Start.

Windows Latest

61,057 Aufrufe • vor 1 Monat

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇

ZEUS⚡️

85,803 Aufrufe • vor 1 Monat

10 YEAR OLD KID FROM AFRICA JUST KILLED HIGGSFIELD, Arcads, MakeUGC and every AI Saas in the market right now. You can now create realistic 4K AI UGC Videos & AI Influencers of similar quality to Seedance2 & Higgsfield But ON YOUR PC, Locally. No monthly Subscription, Local, Offline Arcads Charges $200 a month, Higgsfield Charges $100 a month, credits finish faster than you can count and you they force you to keep on spending. With this you don't need to pay those expensive monthly subscriptions anymore. Easily make 4K Realistic AI UGC Videos for your brands, AI Influencers to post on social media. The Local Studio makes it easy so that beginners can easily run local models on their computer, no comfy ui knowledge needed, and specifically tailored to use cases like AI UGC Videos & AI Influencers. Link & tutorial in the thread👇👇

10 YEAR OLD KID FROM AFRICA JUST KILLED HIGGSFIELD, Arcads, MakeUGC and every AI Saas in the market right now. You can now create realistic 4K AI UGC Videos & AI Influencers of similar quality to Seedance2 & Higgsfield But ON YOUR PC, Locally. No monthly Subscription, Local, Offline Arcads Charges $200 a month, Higgsfield Charges $100 a month, credits finish faster than you can count and you they force you to keep on spending. With this you don't need to pay those expensive monthly subscriptions anymore. Easily make 4K Realistic AI UGC Videos for your brands, AI Influencers to post on social media. The Local Studio makes it easy so that beginners can easily run local models on their computer, no comfy ui knowledge needed, and specifically tailored to use cases like AI UGC Videos & AI Influencers. Link & tutorial in the thread👇👇

Noor

10,809 Aufrufe • vor 2 Tagen

This free model just beat every closed-source AI on coding benchmarks. Open weights. 6x cheaper than Opus. The labs don't want you to know it exists. > GLM-5.2 from Zai just topped Code Arena - the first open-weights model to ever hold the #1 coding spot. Not a leaked weight, not a fine-tune. A fully open model beating GPT and Claude on their own turf. It's live on Hugging Face Inference API right now. 5 providers: Novita, Together AI, Fireworks, Deepinfra, Zai. OpenAI-compatible client. → Go to huggingface(.)co → grab HF_TOKEN from account settings → plug into any OpenAI-compatible client 6x cheaper than Opus. Companies bleeding on AI bills are already routing to this for orchestration, caching, and token optimization. The smart money moved before the headline dropped. > Now you know. huggingface(.)co/zai-org/GLM-5.2 Bookmark this before everyone else figures it out.

This free model just beat every closed-source AI on coding benchmarks. Open weights. 6x cheaper than Opus. The labs don't want you to know it exists. > GLM-5.2 from Zai just topped Code Arena - the first open-weights model to ever hold the #1 coding spot. Not a leaked weight, not a fine-tune. A fully open model beating GPT and Claude on their own turf. It's live on Hugging Face Inference API right now. 5 providers: Novita, Together AI, Fireworks, Deepinfra, Zai. OpenAI-compatible client. → Go to huggingface(.)co → grab HF_TOKEN from account settings → plug into any OpenAI-compatible client 6x cheaper than Opus. Companies bleeding on AI bills are already routing to this for orchestration, caching, and token optimization. The smart money moved before the headline dropped. > Now you know. huggingface(.)co/zai-org/GLM-5.2 Bookmark this before everyone else figures it out.

Atenov int.

12,395 Aufrufe • vor 1 Monat

HOLY MOLY running a 35B model locally on a MacBook shouldn’t be THIS FAST 🤯 Spent my weekend in atomic.chat testing Qwen 35B vs. Qwen 27B on my local machine. I had them generate a fully animated HTML/Canvas car mini-game (demo below), ... and both models breezed through the physics and parallax scrolling without a hitch! The secret sauce here is the Atomic Chat app. Because it's perfectly optimized for Mac and uses Google's new TurboQuant under the hood, you can run heavy open-source models flawlessly while keeping top-tier output quality 👊 Other perks: → ZERO setup required → Access 1,000+ models completely free → 100% offline and private → Zero API limits ... and MUCH more! I dropped the prompt I used in the 🧵↓ Spin it up locally and let me know what you get!

HOLY MOLY running a 35B model locally on a MacBook shouldn’t be THIS FAST 🤯 Spent my weekend in atomic.chat testing Qwen 35B vs. Qwen 27B on my local machine. I had them generate a fully animated HTML/Canvas car mini-game (demo below), ... and both models breezed through the physics and parallax scrolling without a hitch! The secret sauce here is the Atomic Chat app. Because it's perfectly optimized for Mac and uses Google's new TurboQuant under the hood, you can run heavy open-source models flawlessly while keeping top-tier output quality 👊 Other perks: → ZERO setup required → Access 1,000+ models completely free → 100% offline and private → Zero API limits ... and MUCH more! I dropped the prompt I used in the 🧵↓ Spin it up locally and let me know what you get!

Charly Wargnier

100,041 Aufrufe • vor 2 Monaten

Microsoft's hidden Windows 11 trick makes apps launch 70% faster. I tested it on a low-end PC, and early results are promising. Right now, when you click Start, open File Explorer, launch Edge, or right-click for a context menu, and there’s often that tiny micro-stutter before anything happens. Microsoft is now testing a feature called Low Latency Profile. Once turned on, and you do a high-priority action, Windows 11 briefly pushes the CPU to max frequency for 1–3 seconds, finishes the task faster, then drops back down. In my testing on a constrained VM with just 2 cores and 4GB RAM, the difference was obvious. Edge, Outlook, Copilot, and the Start menu opened much faster. CPU usage spiked to around 96–97%, but only for a few seconds. For high-end PCs, the difference may be small. But for budget laptops and low-end Windows 11 machines, this could be a real game-changer.

Windows Latest

927,958 Aufrufe • vor 2 Monaten