Загрузка видео...

Не удалось загрузить видео

На главную

Holy shit... Microsoft open sourced an inference framework that runs a 100B parameter LLM on a single CPU. It's called BitNet. And it does what was supposed to be impossible. No GPU. No cloud. No $10K hardware setup. Just your laptop running a 100-billion parameter model at human reading...

heygurisingh's profile picture

Guri Singh

59,928 subscribers

2,180,357 просмотров • 7 месяцев назад •via X (Twitter)

ОбразованиеЗдоровье и велнесНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

You can now run a 2.78 trillion parameter AI with just 8GB of RAM 😳 kimi-k3-in-c is a FREE, open-source engine that runs the full Kimi K3 model on a single CPU. What you get: > The full 2.78T Kimi K3, not a cut-down version > Runs in 8.24GB of RAM with no GPU > The whole engine is a 176KB C file, with no PyTorch or CUDA > The same output as a 128GB machine, only slower > Chat mode, and it builds on Linux, macOS and Windows How to get it: 1. Clone 2. Run make, then ./scripts/k3-doctor.sh to check your machine 3. Download the checkpoint and pick a preset that fits your RAM It's slow and needs a big SSD, but a trillion parameter model on 8GB of RAM is wild 👀
0:12

You can now run a 2.78 trillion parameter AI with just 8GB of RAM 😳 kimi-k3-in-c is a FREE, open-source engine that runs the full Kimi K3 model on a single CPU. What you get: > The full 2.78T Kimi K3, not a cut-down version > Runs in 8.24GB of RAM with no GPU > The whole engine is a 176KB C file, with no PyTorch or CUDA > The same output as a 128GB machine, only slower > Chat mode, and it builds on Linux, macOS and Windows How to get it: 1. Clone 2. Run make, then ./scripts/k3-doctor.sh to check your machine 3. Download the checkpoint and pick a preset that fits your RAM It's slow and needs a big SSD, but a trillion parameter model on 8GB of RAM is wild 👀

Wojak | AI × Web3 Dev

317,159 просмотров • 2 дней назад

Someone just built a desktop app that that generates 3D models from images and runs 100% locally. It's called Modly. It runs entirely on your GPU, no cloud, no API bills. Just drop an image and get a 3D mesh. 100% Open Source.
0:11

Someone just built a desktop app that that generates 3D models from images and runs 100% locally. It's called Modly. It runs entirely on your GPU, no cloud, no API bills. Just drop an image and get a 3D mesh. 100% Open Source.

How To Prompt

238,264 просмотров • 6 месяцев назад

Google Translate is cooked after this. A developer built a local AI translation engine that runs 40 languages entirely on your own laptop. It's called LibreTranslate. No API key. No usage limits. No sending your documents to Google's servers. You install it once. It runs forever. Here's what it handles: → Paste text. Translated instantly. → Drop in a file. Outputs the translated version. → Point it at a URL. Returns the page in your language. → Build it into your own app via its local REST API. The speed is not the story. The privacy is. Google Translate reads every sentence you paste into it. Legal contracts. Medical records. Internal emails. Client documents. Every word goes to their servers and stays there. LibreTranslate runs entirely offline. Nothing leaves your machine. Ever. The numbers: → 40 languages supported → Runs on CPU -- no GPU needed → Self-hosted in under 5 minutes → REST API built in for developers → 10K+ stars on GitHub 100% open source. MIT licensed. Price: $0. Google charges nothing for Translate either but it charges you something else. GitHub:
0:17

Google Translate is cooked after this. A developer built a local AI translation engine that runs 40 languages entirely on your own laptop. It's called LibreTranslate. No API key. No usage limits. No sending your documents to Google's servers. You install it once. It runs forever. Here's what it handles: → Paste text. Translated instantly. → Drop in a file. Outputs the translated version. → Point it at a URL. Returns the page in your language. → Build it into your own app via its local REST API. The speed is not the story. The privacy is. Google Translate reads every sentence you paste into it. Legal contracts. Medical records. Internal emails. Client documents. Every word goes to their servers and stays there. LibreTranslate runs entirely offline. Nothing leaves your machine. Ever. The numbers: → 40 languages supported → Runs on CPU -- no GPU needed → Self-hosted in under 5 minutes → REST API built in for developers → 10K+ stars on GitHub 100% open source. MIT licensed. Price: $0. Google charges nothing for Translate either but it charges you something else. GitHub:

Rimsha Bhardwaj

89,841 просмотров • 4 месяцев назад

I love this opensource project! Someone just put a 28.9M parameter AI model on an ESP32-S3. A chip that costs only a few dollars. No cloud. No API. No internet. It runs locally at ~9.5 tokens/s and can generate stories on a tiny screen. This changes the way we think about AI hardware. The next generation of AI toys, robots, and IoT devices may not need the cloud at all. Small chips are getting smarter. 🤖
0:11

I love this opensource project! Someone just put a 28.9M parameter AI model on an ESP32-S3. A chip that costs only a few dollars. No cloud. No API. No internet. It runs locally at ~9.5 tokens/s and can generate stories on a tiny screen. This changes the way we think about AI hardware. The next generation of AI toys, robots, and IoT devices may not need the cloud at all. Small chips are getting smarter. 🤖

Steven Cheng

105,352 просмотров • 3 месяцев назад

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.
0:17

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.

Superman

105,212 просмотров • 3 месяцев назад

PewDiePie just hit 20K GitHub stars in under 24 hours. The project? Odysseus. A self-hosted AI workspace that runs 100% on your machine. • Agents with tools • MCP built in • Persistent memory • File handling • Windows, macOS, Linux Your data never leaves your device. It supports Ollama, llama.cpp, and vLLM locally with OpenAI and OpenRouter support if you want cloud models too. The crazy part? A YouTuber with 110M+ subscribers just out-shipped most AI startups. And he built half of it using AI.
0:12

PewDiePie just hit 20K GitHub stars in under 24 hours. The project? Odysseus. A self-hosted AI workspace that runs 100% on your machine. • Agents with tools • MCP built in • Persistent memory • File handling • Windows, macOS, Linux Your data never leaves your device. It supports Ollama, llama.cpp, and vLLM locally with OpenAI and OpenRouter support if you want cloud models too. The crazy part? A YouTuber with 110M+ subscribers just out-shipped most AI startups. And he built half of it using AI.

Charlie Hills

16,546 просмотров • 5 месяцев назад

Cancelled ChatGPT -> Built JARVIS -> Pays $0 -> it works offline + it's smarter than the $20/month version. No WiFi needed, no cloud, no API keys, no rate limits, no queues, no $20/month just to ask a server in Virginia for the weather. Just a local model running directly on the laptop hardware, voice activated, system integrated, controlling apps, answering questions, doing the work. Iron Man had JARVIS embedded in his suit, this guy has it embedded in his MacBook and it works on a plane, in a basement, on a remote cabin with zero signal. OpenAI is burning $700,000 a day on infrastructure to deliver something this guy runs for free. Anthropic charges $200/month for unlimited Claude access, microsoft built Copilot into every product they sell. This guy skipped all of it, downloaded a model and made his laptop the smartest device in the room. No subscription. No login. No internet. No data sent anywhere ever. The most powerful AI assistant on earth is now the one running locally on hardware you already own. ChatGPT charges you to think slower, he pays nothing and thinks alone, he made it himself.
0:15

Cancelled ChatGPT -> Built JARVIS -> Pays $0 -> it works offline + it's smarter than the $20/month version. No WiFi needed, no cloud, no API keys, no rate limits, no queues, no $20/month just to ask a server in Virginia for the weather. Just a local model running directly on the laptop hardware, voice activated, system integrated, controlling apps, answering questions, doing the work. Iron Man had JARVIS embedded in his suit, this guy has it embedded in his MacBook and it works on a plane, in a basement, on a remote cabin with zero signal. OpenAI is burning $700,000 a day on infrastructure to deliver something this guy runs for free. Anthropic charges $200/month for unlimited Claude access, microsoft built Copilot into every product they sell. This guy skipped all of it, downloaded a model and made his laptop the smartest device in the room. No subscription. No login. No internet. No data sent anywhere ever. The most powerful AI assistant on earth is now the one running locally on hardware you already own. ChatGPT charges you to think slower, he pays nothing and thinks alone, he made it himself.

Defileo🔮

154,800 просмотров • 6 месяцев назад

🚨 Alibaba just open sourced a GUI agent that lives inside your webpage and controls it with natural language. It's called Page Agent and it's not a browser extension. It's pure JavaScript no Python, no Puppeteer, no headless browser, no screenshots. Just one script tag and your web app understands natural language. Here's what it actually does: → Embed it with a single tag or npm install → Control any web interface with plain English commands → Text-based DOM manipulation no OCR, no vision models needed → Bring your own LLM (GPT, Claude, Qwen, anything) → Ships a built-in UI with human-in-the-loop support → Turn 20-click ERP/CRM workflows into one sentence → Optional Chrome extension for multi-tab agent tasks → Works on any web app SaaS, admin panels, internal tools Companies are charging $30/month for AI copilots built on this exact idea. This is 3 lines of code. Your users. Your interface. The AI copilot layer for every web app just got open sourced. 1.6K stars. 100% Open Source. (Link in the comments)
0:18

🚨 Alibaba just open sourced a GUI agent that lives inside your webpage and controls it with natural language. It's called Page Agent and it's not a browser extension. It's pure JavaScript no Python, no Puppeteer, no headless browser, no screenshots. Just one script tag and your web app understands natural language. Here's what it actually does: → Embed it with a single tag or npm install → Control any web interface with plain English commands → Text-based DOM manipulation no OCR, no vision models needed → Bring your own LLM (GPT, Claude, Qwen, anything) → Ships a built-in UI with human-in-the-loop support → Turn 20-click ERP/CRM workflows into one sentence → Optional Chrome extension for multi-tab agent tasks → Works on any web app SaaS, admin panels, internal tools Companies are charging $30/month for AI copilots built on this exact idea. This is 3 lines of code. Your users. Your interface. The AI copilot layer for every web app just got open sourced. 1.6K stars. 100% Open Source. (Link in the comments)

Ihtesham Ali

135,634 просмотров • 7 месяцев назад

HERMES AGENT GEMMA 4 26B ON A MAC MINI NO API KEYS NO MONTHLY BILL NO ASKING PERMISSION TO USE YOUR OWN DATA a $700 box on your desk just replaced a $30/month subscription, runs a 26b model that outperforms what used to need a data center, texts you on telegram and never forgets a single thing you taught it the gap between people who own their ai and people who rent it is about to get very wide full breakdown on how this actually works ↓
0:15

HERMES AGENT GEMMA 4 26B ON A MAC MINI NO API KEYS NO MONTHLY BILL NO ASKING PERMISSION TO USE YOUR OWN DATA a $700 box on your desk just replaced a $30/month subscription, runs a 26b model that outperforms what used to need a data center, texts you on telegram and never forgets a single thing you taught it the gap between people who own their ai and people who rent it is about to get very wide full breakdown on how this actually works ↓

wandermist

158,628 просмотров • 5 месяцев назад

Your PC can now run an entire AI studio for free. Locally Uncensored puts chat, image generation, video generation, and a coding agent in one desktop app for Windows and Linux. - Chat with local models and your own documents - Generate images and videos on your GPU - Write code and review changes before applying them - Run agents with tools and MCP connections Install it, pick a model, and start. It shows which models fit your hardware before you download. No Docker. No terminal. No cloud required for the local workflows. 100% Free. Open Source.
0:15

Your PC can now run an entire AI studio for free. Locally Uncensored puts chat, image generation, video generation, and a coding agent in one desktop app for Windows and Linux. - Chat with local models and your own documents - Generate images and videos on your GPU - Write code and review changes before applying them - Run agents with tools and MCP connections Install it, pick a model, and start. It shows which models fit your hardware before you download. No Docker. No terminal. No cloud required for the local workflows. 100% Free. Open Source.

Oliver Prompts

21,751 просмотров • 28 дней назад

this is the worst local AI will ever be. tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.
0:17

this is the worst local AI will ever be. tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.

Sudo su

106,822 просмотров • 8 месяцев назад

90% of "AI developers" just download pre packaged GGUF files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.
0:19

90% of "AI developers" just download pre packaged GGUF files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.

Alok

62,631 просмотров • 3 месяцев назад

🚨 AN UNCENSORED AI MODEL THAT FITS ON YOUR PHONE💀💀 Meet LFM3-1.7B Uncensored a lightweight local AI model designed to run on everyday hardware. What makes it interesting: Around 1 GB for a quantized version Up to 256K context supported by the model family Runs locally without a cloud API No monthly subscription or per-token fees Community-tuned for fewer content restrictions You can experiment with it using compatible local AI apps even on a phone with enough available memory. Note: The 1 GB figure depends on quantization, and actual performance varies by device. This specific model is text-only; vision and audio require separate models. 🔗 Model:
0:18

🚨 AN UNCENSORED AI MODEL THAT FITS ON YOUR PHONE💀💀 Meet LFM3-1.7B Uncensored a lightweight local AI model designed to run on everyday hardware. What makes it interesting: Around 1 GB for a quantized version Up to 256K context supported by the model family Runs locally without a cloud API No monthly subscription or per-token fees Community-tuned for fewer content restrictions You can experiment with it using compatible local AI apps even on a phone with enough available memory. Note: The 1 GB figure depends on quantization, and actual performance varies by device. This specific model is text-only; vision and audio require separate models. 🔗 Model:

Forha

25,527 просмотров • 6 дней назад

China just dropped an open source model that turns any phone into a 3D scanner you point it at a room and walk through it the whole space builds itself into a 3D point cloud in real time as the camera moves, no LiDAR and no depth sensor anywhere it held a full 13 minute walkthrough at 25,000 frames without the map drifting or collapsing > ~20 fps on a single GPU > works indoors and outdoors from plain footage > the scan normally needs a $50,000 rig and a technician > Apache 2.0 and runs fully offline on your machine repo in the reply
0:18

China just dropped an open source model that turns any phone into a 3D scanner you point it at a room and walk through it the whole space builds itself into a 3D point cloud in real time as the camera moves, no LiDAR and no depth sensor anywhere it held a full 13 minute walkthrough at 25,000 frames without the map drifting or collapsing > ~20 fps on a single GPU > works indoors and outdoors from plain footage > the scan normally needs a $50,000 rig and a technician > Apache 2.0 and runs fully offline on your machine repo in the reply

savip.

105,419 просмотров • 3 месяцев назад

Introducing Pods Hyperspace Pods lets a small group of people - a family, a startup, a few friends, to pool their laptops and desktops into one AI cluster. Everyone installs the CLI, someone creates a pod, shares an invite link, and the machines form a mesh. Models like Qwen 3.5 32B or GLM-5 Turbo that need more memory than any single laptop has get automatically sharded across the group's devices - layers split proportionally, inference pipelined through the ring. From the outside it looks like one OpenAI-compatible API endpoint with a pk_* key that drops straight into your AI tools and products. No configuration beyond pasting the key and changing the base URL. A team of five paying for cloud AI burns $500–2,000 a month on API calls. The same team's existing machines can serve Qwen 3.5 (competitive on SWE-bench) and GLM-5 Turbo (#1 on BrowseComp for tool-calling and web research) for free - the hardware is already on their desks. When a query genuinely needs a frontier model nobody has locally, the pod falls back to cloud at wholesale rates from a shared treasury. But for the daily work - code reviews, refactors, research, drafting - local models handle it and nobody gets billed. And when it is idle, you can rent out your pod on the compute marketplace, with fine-grained permissions for access management. There's no central server involved in inference. Prompts go from your machine to your pod members' machines and back: all of this enabled by the fully peer-to-peer Hyperspace network. Pod state - who's a member, which API keys are valid, how much treasury is left - is replicated across members with consensus, so the whole thing works on a local network. Members behind home routers don't need port forwarding either. The practical setup for most pods is three models covering different jobs: Qwen 3.5 32B for code and reasoning, GLM-5 Turbo for browsing and research, Gemma 4 for fast lightweight tasks. All running on hardware you already own. Pods ship today in Hyperspace v5.19. Model sharding, API keys, treasury, and Raft coordinator are all live. What Makes This Different - No middleman. Your prompts travel from your IDE to your pod members' hardware and back. There is no server in between reading your data. - No vendor lock-in. Pod membership, API keys, and treasury are replicated across your own machines using Raft consensus. If the internet goes down, your local network keeps working. There is no database in someone else's cloud that your pod depends on. - Automatic sharding. You don't configure layer ranges or calculate VRAM budgets. Tell the pod which model you want. It figures out how to split it across whatever hardware is online. - Real NAT traversal. Your friend behind a home router with a dynamic IP? Works. No VPN, no Tailscale, no port forwarding. The nodes handle it. - Free when local. This is the part that matters most. Cloud AI bills scale with usage. Pod inference on local hardware scales with nothing. The marginal cost of your 10,000th prompt is the electricity your laptop was already using. Coming soon: - Pod federation: pods form alliances with other pods. - Marketplace: pods with spare capacity can sell inference to other pods.
0:19

Introducing Pods Hyperspace Pods lets a small group of people - a family, a startup, a few friends, to pool their laptops and desktops into one AI cluster. Everyone installs the CLI, someone creates a pod, shares an invite link, and the machines form a mesh. Models like Qwen 3.5 32B or GLM-5 Turbo that need more memory than any single laptop has get automatically sharded across the group's devices - layers split proportionally, inference pipelined through the ring. From the outside it looks like one OpenAI-compatible API endpoint with a pk_* key that drops straight into your AI tools and products. No configuration beyond pasting the key and changing the base URL. A team of five paying for cloud AI burns $500–2,000 a month on API calls. The same team's existing machines can serve Qwen 3.5 (competitive on SWE-bench) and GLM-5 Turbo (#1 on BrowseComp for tool-calling and web research) for free - the hardware is already on their desks. When a query genuinely needs a frontier model nobody has locally, the pod falls back to cloud at wholesale rates from a shared treasury. But for the daily work - code reviews, refactors, research, drafting - local models handle it and nobody gets billed. And when it is idle, you can rent out your pod on the compute marketplace, with fine-grained permissions for access management. There's no central server involved in inference. Prompts go from your machine to your pod members' machines and back: all of this enabled by the fully peer-to-peer Hyperspace network. Pod state - who's a member, which API keys are valid, how much treasury is left - is replicated across members with consensus, so the whole thing works on a local network. Members behind home routers don't need port forwarding either. The practical setup for most pods is three models covering different jobs: Qwen 3.5 32B for code and reasoning, GLM-5 Turbo for browsing and research, Gemma 4 for fast lightweight tasks. All running on hardware you already own. Pods ship today in Hyperspace v5.19. Model sharding, API keys, treasury, and Raft coordinator are all live. What Makes This Different - No middleman. Your prompts travel from your IDE to your pod members' hardware and back. There is no server in between reading your data. - No vendor lock-in. Pod membership, API keys, and treasury are replicated across your own machines using Raft consensus. If the internet goes down, your local network keeps working. There is no database in someone else's cloud that your pod depends on. - Automatic sharding. You don't configure layer ranges or calculate VRAM budgets. Tell the pod which model you want. It figures out how to split it across whatever hardware is online. - Real NAT traversal. Your friend behind a home router with a dynamic IP? Works. No VPN, no Tailscale, no port forwarding. The nodes handle it. - Free when local. This is the part that matters most. Cloud AI bills scale with usage. Pod inference on local hardware scales with nothing. The marginal cost of your 10,000th prompt is the electricity your laptop was already using. Coming soon: - Pod federation: pods form alliances with other pods. - Marketplace: pods with spare capacity can sell inference to other pods.

Varun

310,994 просмотров • 6 месяцев назад

You don't need a GPU for fast studio grade voice cloning anymore. Qwen3 TTS (1.7B Q4_K_M) + mainline llama.cpp is officially the fastest way to generate zero shot voice clones using 100% pure CPU execution. Following up on my last post where we ran the Q8 model on a GPU, we just took local C++ voice synthesis a massive step further. The open source community quantized Alibaba's SOTA Qwen3 TTS model down to Q4_K_M GGUF, completely freeing local audio pipelines from dedicated graphics hardware. Here is the real world benchmark and hardware breakdown of running SOTA voice cloning on CPU: # Architecture & Model Setup Using Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf paired with the 8 bit multimodal projector (mmproj-Q8_0.gguf), llama.cpp executes the entire pipeline in pure C++. No PyTorch, no CUDA dependencies, and no VRAM bottlenecks. # Real-World Memory Footprint - Baseline RAM: 1.6 GB system idle. - Peak Generation RAM: 8 GB RAM during active voice synthesis. - Requirement: Any basic machine with at least 8 GB of system RAM can run this easily. # Real World CPU Benchmarks - Google Colab Free Tier (Throttled 2 Core CPU): Synthesizes a 5 sec studio quality audio clip (~8 words) in 45 seconds. - Modern Consumer CPU (Intel i5/i7 13th/14th Gen or AMD Ryzen 7000/9000): generation should drop to 5 to 20 seconds (nearly 1:1 real-time generation speed!). # Zero Shot Voice Cloning Quality Pass any 5 to 20 second .wav audio sample to the C++ engine using the --tts-speaker-file flag. It yields clean, natural sounding cloned speech with virtually zero quality loss compared to unquantized FP16 weights. To make testing seamless, I built an updated zero config Google Colab notebook. It pulls the official pre built llama.cpp CPU binaries (zero compilation time!) launches a live Gradio web app right in your browser. Record a 5 second clip from your mic (or drop a .mp3, .wav file), type text, and generate cloned audio on CPU. Native C++ audio models are making edge based, offline AI voice agents a reality. Links to the free Q4 CPU Colab notebook and the Q4_K_M GGUF HuggingFace repository are in the replies below! Which models have you been running on your CPUs? What CPU hardware are you using for local inference?
0:16

You don't need a GPU for fast studio grade voice cloning anymore. Qwen3 TTS (1.7B Q4_K_M) + mainline llama.cpp is officially the fastest way to generate zero shot voice clones using 100% pure CPU execution. Following up on my last post where we ran the Q8 model on a GPU, we just took local C++ voice synthesis a massive step further. The open source community quantized Alibaba's SOTA Qwen3 TTS model down to Q4_K_M GGUF, completely freeing local audio pipelines from dedicated graphics hardware. Here is the real world benchmark and hardware breakdown of running SOTA voice cloning on CPU: # Architecture & Model Setup Using Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf paired with the 8 bit multimodal projector (mmproj-Q8_0.gguf), llama.cpp executes the entire pipeline in pure C++. No PyTorch, no CUDA dependencies, and no VRAM bottlenecks. # Real-World Memory Footprint - Baseline RAM: 1.6 GB system idle. - Peak Generation RAM: 8 GB RAM during active voice synthesis. - Requirement: Any basic machine with at least 8 GB of system RAM can run this easily. # Real World CPU Benchmarks - Google Colab Free Tier (Throttled 2 Core CPU): Synthesizes a 5 sec studio quality audio clip (~8 words) in 45 seconds. - Modern Consumer CPU (Intel i5/i7 13th/14th Gen or AMD Ryzen 7000/9000): generation should drop to 5 to 20 seconds (nearly 1:1 real-time generation speed!). # Zero Shot Voice Cloning Quality Pass any 5 to 20 second .wav audio sample to the C++ engine using the --tts-speaker-file flag. It yields clean, natural sounding cloned speech with virtually zero quality loss compared to unquantized FP16 weights. To make testing seamless, I built an updated zero config Google Colab notebook. It pulls the official pre built llama.cpp CPU binaries (zero compilation time!) launches a live Gradio web app right in your browser. Record a 5 second clip from your mic (or drop a .mp3, .wav file), type text, and generate cloned audio on CPU. Native C++ audio models are making edge based, offline AI voice agents a reality. Links to the free Q4 CPU Colab notebook and the Q4_K_M GGUF HuggingFace repository are in the replies below! Which models have you been running on your CPUs? What CPU hardware are you using for local inference?

Alok

105,756 просмотров • 2 месяцев назад

ANDREW NG JUST OPEN-SOURCED HIS OWN AI COWORKER. Stanford CS adjunct faculty. former head of Google Brain. 1,758,130 people follow him for AI. the repo is called OpenWorker. what comes back isn't a chat - it's finished work. 13,267 stars. 17 days old. MIT. what it does: - you name the outcome you want - it breaks that into steps and works across your own files - 25+ integrations - mention OpenWorker and a session opens on your desktop, the answer comes back in the thread - runs on a schedule - any model: OpenAI, Anthropic, Gemini, DeepSeek, Kimi, Grok and so on your keys, tokens and conversations stay on your machine. the only cloud piece brokers OAuth. -> everyone else is selling you an assistant that lives at their place. this one lives at yours. save this before the next task you were about to do by hand.
0:15

ANDREW NG JUST OPEN-SOURCED HIS OWN AI COWORKER. Stanford CS adjunct faculty. former head of Google Brain. 1,758,130 people follow him for AI. the repo is called OpenWorker. what comes back isn't a chat - it's finished work. 13,267 stars. 17 days old. MIT. what it does: - you name the outcome you want - it breaks that into steps and works across your own files - 25+ integrations - mention OpenWorker and a session opens on your desktop, the answer comes back in the thread - runs on a schedule - any model: OpenAI, Anthropic, Gemini, DeepSeek, Kimi, Grok and so on your keys, tokens and conversations stay on your machine. the only cloud piece brokers OAuth. -> everyone else is selling you an assistant that lives at their place. this one lives at yours. save this before the next task you were about to do by hand.

Granite

34,256 просмотров • 2 месяцев назад

Right now, you may not have access to models like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.
0:19

Right now, you may not have access to models like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.

Alok

65,387 просмотров • 4 месяцев назад

This tool is literally Higgsfield AI but FREE for good. It's called Wan2GP. A full AI video studio built specifically for people without expensive hardware. Runs on as little as 6GB of VRAM, even old RTX 10-series cards and 8GB laptops. Everything stays on your machine, no uploads, no caps, no watermarks. What you get in one app: • Text-to-video and image-to-video generation • The best open models built in: Wan 2.2, LTX-2, Hunyuan Video, Flux • A full browser interface with a queue system • LoRA support to customize any model • Mask editor and prompt enhancer included A 5-second clip generates in minutes on a mid-range gaming rig. No subscription, ever. 100% Free. Open Source.
0:15

This tool is literally Higgsfield AI but FREE for good. It's called Wan2GP. A full AI video studio built specifically for people without expensive hardware. Runs on as little as 6GB of VRAM, even old RTX 10-series cards and 8GB laptops. Everything stays on your machine, no uploads, no caps, no watermarks. What you get in one app: • Text-to-video and image-to-video generation • The best open models built in: Wan 2.2, LTX-2, Hunyuan Video, Flux • A full browser interface with a queue system • LoRA support to customize any model • Mask editor and prompt enhancer included A 5-second clip generates in minutes on a mid-range gaming rig. No subscription, ever. 100% Free. Open Source.

Simplifying AI

182,717 просмотров • 3 месяцев назад

Meet Husky: a Model-Specific Inference (MSI) engine up to 4.5× faster than Apple's MLX Woof, Underdog's Pareto frontier model, now runs up to 730 tokens/sec on a MacBook Finally local models are as fast & capable. Try it now in - your personal private AI
0:19

Meet Husky: a Model-Specific Inference (MSI) engine up to 4.5× faster than Apple's MLX Woof, Underdog's Pareto frontier model, now runs up to 730 tokens/sec on a MacBook Finally local models are as fast & capable. Try it now in - your personal private AI

Sigil Wen

436,286 просмотров • 1 месяц назад