Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

local LLMs on Mac are getting good. but the workflow around models is still messy. you have models in: •⁠ ⁠LM Studio •⁠ ⁠Ollama •⁠ ⁠Hugging Face cache •⁠ ⁠random folders •⁠ ⁠manual downloads and every runtime expects something slightly different. we built ModelHub to clean up this layer....

sabeshbharathi's profile picture

sabesh 📟

2,269 subscribers

31,497 Aufrufe • vor 3 Monaten •via X (Twitter)

Nachrichten & PolitikBildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

a guy put a local AI coding stack on an Android phone Termux on the screen Ollama underneath a local model answering from the device that tiny terminal is the shift. two years ago, local LLMs felt like punishment fans screaming bad models one token per second cloud looked inevitable now the same idea runs almost anywhere. - Raspberry Pi for tiny chat - MacBook Air for daily writing - RTX 3090 for serious inference - LM Studio when you want the clean GUI - Ollama when you want apps to connect the point is not replacing GPT-5.1 or Opus 4.8 on the hardest tasks. the point is owning the boring 80%: - drafts - summaries - code help - private docs - offline workflows no token anxiety no rate-limit ceiling no sensitive files leaving the machine cloud AI became the premium layer. local AI became the workbench.
0:26

a guy put a local AI coding stack on an Android phone Termux on the screen Ollama underneath a local model answering from the device that tiny terminal is the shift. two years ago, local LLMs felt like punishment fans screaming bad models one token per second cloud looked inevitable now the same idea runs almost anywhere. - Raspberry Pi for tiny chat - MacBook Air for daily writing - RTX 3090 for serious inference - LM Studio when you want the clean GUI - Ollama when you want apps to connect the point is not replacing GPT-5.1 or Opus 4.8 on the hardest tasks. the point is owning the boring 80%: - drafts - summaries - code help - private docs - offline workflows no token anxiety no rate-limit ceiling no sensitive files leaving the machine cloud AI became the premium layer. local AI became the workbench.

Myttle

27,108 Aufrufe • vor 2 Monaten

Create your own local and free AI agent using an open source model You can combine: - IBM's Granite 3.3 8B AI model - LM Studio to run it on a laptop - Smolagents to build your agent Small AI models are now powerful enough to run an autonomous agent. Thanks to IBM France for sponsoring this content!
8:44

Create your own local and free AI agent using an open source model You can combine: - IBM's Granite 3.3 8B AI model - LM Studio to run it on a laptop - Smolagents to build your agent Small AI models are now powerful enough to run an autonomous agent. Thanks to IBM France for sponsoring this content!

Paul Couvert

18,890 Aufrufe • vor 1 Jahr

i tested a fully local ai coding setup: gemma4 from Google DeepMind running through LM Studio, paired with pi which is a minimal open source coding harness this is part 1 of a two part series. here we cover the why and the complete hands-on setup 0:00 - Running a coding agent fully local 0:47 - Why Pi as the coding agent harness 1:39 - Why Gemma 4 and which variant to pick 2:44 - Architecture: Pi, LM Studio and Gemma 4 3:45 - LM Studio setup and model download 5:01 - Starting the local server 5:19 - API sanity check with curl 6:08 - Installing the Pi coding agent 6:20 - Configuring models.json for local models 8:20 - First run: selecting the model in Pi 8:58 - Fixing the context length error 10:16 - Testing the working setup 10:50 - What is coming in Part 2
11:32

i tested a fully local ai coding setup: gemma4 from Google DeepMind running through LM Studio, paired with pi which is a minimal open source coding harness this is part 1 of a two part series. here we cover the why and the complete hands-on setup 0:00 - Running a coding agent fully local 0:47 - Why Pi as the coding agent harness 1:39 - Why Gemma 4 and which variant to pick 2:44 - Architecture: Pi, LM Studio and Gemma 4 3:45 - LM Studio setup and model download 5:01 - Starting the local server 5:19 - API sanity check with curl 6:08 - Installing the Pi coding agent 6:20 - Configuring models.json for local models 8:20 - First run: selecting the model in Pi 8:58 - Fixing the context length error 10:16 - Testing the working setup 10:50 - What is coming in Part 2

Nidhi Singh

16,289 Aufrufe • vor 2 Monaten

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.
0:26

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Harper Carroll

30,392 Aufrufe • vor 19 Tagen

GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.
1:52

GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.

Alex Cheema

30,800 Aufrufe • vor 1 Monat

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits​​​​​​​​​​​​​​​​ full breakdown of my raw llama.cpp setup ↓
21:03

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits​​​​​​​​​​​​​​​​ full breakdown of my raw llama.cpp setup ↓

leopardracer

238,950 Aufrufe • vor 3 Monaten

Laguna S 2.1 is built to run locally. We heard the ask for a serious open model in the 80–120B range. Laguna S 2.1 is our answer, and we want it to redefine what this size class can do. Weights ship in BF16, FP8, INT4 and NVFP4, with official GGUF and MLX conversions, plus official DFlash draft models for faster inference. Day-one support across llama.cpp, ollama, Valerie Lee Make Up, SGLang, NVIDIA TRT-LLM, Hugging Face transformers and ZML. Watch it run on an NVIDIA AI DGX Spark.
0:39

Laguna S 2.1 is built to run locally. We heard the ask for a serious open model in the 80–120B range. Laguna S 2.1 is our answer, and we want it to redefine what this size class can do. Weights ship in BF16, FP8, INT4 and NVFP4, with official GGUF and MLX conversions, plus official DFlash draft models for faster inference. Day-one support across llama.cpp, ollama, Valerie Lee Make Up, SGLang, NVIDIA TRT-LLM, Hugging Face transformers and ZML. Watch it run on an NVIDIA AI DGX Spark.

Poolside

31,242 Aufrufe • vor 1 Monat

🐆 Rapid-MLX v0.12 is here. We’ve officially evolved from a simple chat app into a full-fledged, on-device AI studio for Apple Silicon! 🖥️✨ We didn't just push the MLX inference engine to its limits and expand support for a massive lineup of local open-source models—we are alpha-launching the highly anticipated Desktop Version. (A huge shoutout to the IoTeX community for grinding through the closed beta with us. Your feedback was incredible and helped shape this beast.) Here are the game-changing features you can run on your Mac right now, 100% free and 100% offline 👇 🚀 Blazing Fast Local LLMs Run anything from 4B up to Qwen3.5-122B completely offline. No guessing games—we recommend models matched perfectly to your Mac's actual RAM. Rich chat includes syntax highlighting, markdown tables, and honest tok/s metrics. 🎨 Local Image Generation A brand new Images tab to render directly on your machine. Pick a model (FLUX.2-klein, Z-Image-Turbo), prompt, and refine. Everything lands in a visual filmstrip. 👁️ Vision & Live Web Tools Attach an image and chat about it with local vision. Need real-time data? Our built-in web tools (weather, search, page-fetch) run mid-answer with strict, transparent privacy controls. 🤖 Plug-and-Play Coding Agents Wire up Claude Code, Codex, Cline, or Continue in seconds. One copy-paste from the Launch tab spins up a local OpenAI/Anthropic-compatible endpoint. 🔒 Private by Design Everything runs on-device. Signed, notarized, and entirely local. Your data stays yours. Turn your Mac into an AI powerhouse today. ⚡️
0:37

🐆 Rapid-MLX v0.12 is here. We’ve officially evolved from a simple chat app into a full-fledged, on-device AI studio for Apple Silicon! 🖥️✨ We didn't just push the MLX inference engine to its limits and expand support for a massive lineup of local open-source models—we are alpha-launching the highly anticipated Desktop Version. (A huge shoutout to the IoTeX community for grinding through the closed beta with us. Your feedback was incredible and helped shape this beast.) Here are the game-changing features you can run on your Mac right now, 100% free and 100% offline 👇 🚀 Blazing Fast Local LLMs Run anything from 4B up to Qwen3.5-122B completely offline. No guessing games—we recommend models matched perfectly to your Mac's actual RAM. Rich chat includes syntax highlighting, markdown tables, and honest tok/s metrics. 🎨 Local Image Generation A brand new Images tab to render directly on your machine. Pick a model (FLUX.2-klein, Z-Image-Turbo), prompt, and refine. Everything lands in a visual filmstrip. 👁️ Vision & Live Web Tools Attach an image and chat about it with local vision. Need real-time data? Our built-in web tools (weather, search, page-fetch) run mid-answer with strict, transparent privacy controls. 🤖 Plug-and-Play Coding Agents Wire up Claude Code, Codex, Cline, or Continue in seconds. One copy-paste from the Launch tab spins up a local OpenAI/Anthropic-compatible endpoint. 🔒 Private by Design Everything runs on-device. Signed, notarized, and entirely local. Your data stays yours. Turn your Mac into an AI powerhouse today. ⚡️

raullen

33,889 Aufrufe • vor 12 Tagen

Today, we’re launching Gemma 4, our most intelligent open models to date. Built with the same breakthrough technology as Gemini 3, Gemma 4 brings advanced reasoning to your personal hardware and devices. Here’s what Gemma 4 unlocks for developers: — Intelligence-per-parameter: Our 31B (Dense) and 26B (MoE) models deliver state-of-the-art performance for their size, outcompeting models 20x their size on Arena.ai — Commercial flexibility: Released under a permissive Apache 2.0 license for complete developer flexibility and digital sovereignty — Agentic workflows: Native support for function-calling and structured JSON output allows you to build reliable, autonomous agents — Multimodal edge AI: The E2B and E4B models bring native vision, audio, and low latency to mobile and IoT devices — Long-context reasoning: Up to 256K context windows allow you to process entire repositories or large documents in a single prompt Whether you're building global applications in 140+ languages or local-first AI code assistants, Gemma 4 is built to be your foundation. Explore in Google AI Studio or download the weights on Hugging Face, Kaggle, and ollama.
2:51

Today, we’re launching Gemma 4, our most intelligent open models to date. Built with the same breakthrough technology as Gemini 3, Gemma 4 brings advanced reasoning to your personal hardware and devices. Here’s what Gemma 4 unlocks for developers: — Intelligence-per-parameter: Our 31B (Dense) and 26B (MoE) models deliver state-of-the-art performance for their size, outcompeting models 20x their size on Arena.ai — Commercial flexibility: Released under a permissive Apache 2.0 license for complete developer flexibility and digital sovereignty — Agentic workflows: Native support for function-calling and structured JSON output allows you to build reliable, autonomous agents — Multimodal edge AI: The E2B and E4B models bring native vision, audio, and low latency to mobile and IoT devices — Long-context reasoning: Up to 256K context windows allow you to process entire repositories or large documents in a single prompt Whether you're building global applications in 140+ languages or local-first AI code assistants, Gemma 4 is built to be your foundation. Explore in Google AI Studio or download the weights on Hugging Face, Kaggle, and ollama.

Google AI

1,670,290 Aufrufe • vor 4 Monaten

✨ Personal AI is the next computing platform. AI is shifting from something you access to something you build with, locally, at the edge, and across systems. We’re unlocking new possibilities for developers: • AMD Ryzen AI Halo, a local-first developer system, preorder starting in June, develop AI without limits on your desk • Gorgon Halo with up to 192GB unified memory, supporting 300B+ parameter models locally, run massive models locally We’re excited to partner with clem 🤗🤗, Co-founder and CEO of Hugging Face, to advance open-source AI for Ryzen AI. Our focus is seamless AI, from model to deployment. Cloud, edge, device. One continuum. Multi-agent systems, local inference at scale, open models as infrastructure. This is the next computing era 🚀
4:49

✨ Personal AI is the next computing platform. AI is shifting from something you access to something you build with, locally, at the edge, and across systems. We’re unlocking new possibilities for developers: • AMD Ryzen AI Halo, a local-first developer system, preorder starting in June, develop AI without limits on your desk • Gorgon Halo with up to 192GB unified memory, supporting 300B+ parameter models locally, run massive models locally We’re excited to partner with clem 🤗🤗, Co-founder and CEO of Hugging Face, to advance open-source AI for Ryzen AI. Our focus is seamless AI, from model to deployment. Cloud, edge, device. One continuum. Multi-agent systems, local inference at scale, open models as infrastructure. This is the next computing era 🚀

Jack Huynh

49,264 Aufrufe • vor 3 Monaten

MiniMax-M2.1 running fully local in AWQ-4Bit with full context window (170 GB VRAM w full context) - 1000~ to 16,000~ tps prefill - 100~ tps generation speeds - Opencode It’s doing real work, updating my blog with little steering or specificity. The problem with local LLMs is that they require too much steering, this means baby sitting which I don’t have the time to do MiniMax cracked the cost, intelligence, and speed challenge, I would say this is a top tier model. I run frontier models like Gemini and it just fails to call tools, in this year lol… ——————— I think glm-4.?-air is needed still. We need a viable model at each hardware entry point, a Mac M1 Ultra 192GB? is relatively cheap 5k to be able to run this model at 40 tps is a huge societal unlock. Smaller models can be good but size matters :p
3:01

MiniMax-M2.1 running fully local in AWQ-4Bit with full context window (170 GB VRAM w full context) - 1000~ to 16,000~ tps prefill - 100~ tps generation speeds - Opencode It’s doing real work, updating my blog with little steering or specificity. The problem with local LLMs is that they require too much steering, this means baby sitting which I don’t have the time to do MiniMax cracked the cost, intelligence, and speed challenge, I would say this is a top tier model. I run frontier models like Gemini and it just fails to call tools, in this year lol… ——————— I think glm-4.?-air is needed still. We need a viable model at each hardware entry point, a Mac M1 Ultra 192GB? is relatively cheap 5k to be able to run this model at 40 tps is a huge societal unlock. Smaller models can be good but size matters :p

0xSero

23,836 Aufrufe • vor 7 Monaten

Today we're launching Accomplish FREE - powered by our new hybrid model router. Since we launched Accomplish a few weeks ago, we've been blown away by what users are building with it. Hundreds of thousands of you downloaded the app and took it for a spin, but one thing kept coming up: not everyone wants to bring their own API key just to get started. So today we're fixing that with free, built-in models - made possible by a massive shift that's happened in just the past few weeks: the rise of fully hosted open-weight models - locally on your Windows machine with NVIDIA NVIDIA GeForce, on your Mac with Apple MLX, or on Accomplish cloud, for FREE. Our new hybrid routing algorithm dynamically routes between cloud models and models running locally on your machine - optimizing for local execution by automatically detecting your hardware capabilities and each sub-task's complexity: coding, visuals, simple classifications - every LLM call is routed to the best model. We also brought some of our favorite enterprise features to the free tier: scheduled task dispatch, Google Workspace integration (Google Drive Docs, Sheets, Slides) via the new Google Workspace CLI, native Slack MCP connectivity, and more. Accomplish FREE is available for macOS, Windows, and Linux. Download, send a task - and boom, it just works with ZERO configuration! Download link in bio / first comment >>
1:54

Today we're launching Accomplish FREE - powered by our new hybrid model router. Since we launched Accomplish a few weeks ago, we've been blown away by what users are building with it. Hundreds of thousands of you downloaded the app and took it for a spin, but one thing kept coming up: not everyone wants to bring their own API key just to get started. So today we're fixing that with free, built-in models - made possible by a massive shift that's happened in just the past few weeks: the rise of fully hosted open-weight models - locally on your Windows machine with NVIDIA NVIDIA GeForce, on your Mac with Apple MLX, or on Accomplish cloud, for FREE. Our new hybrid routing algorithm dynamically routes between cloud models and models running locally on your machine - optimizing for local execution by automatically detecting your hardware capabilities and each sub-task's complexity: coding, visuals, simple classifications - every LLM call is routed to the best model. We also brought some of our favorite enterprise features to the free tier: scheduled task dispatch, Google Workspace integration (Google Drive Docs, Sheets, Slides) via the new Google Workspace CLI, native Slack MCP connectivity, and more. Accomplish FREE is available for macOS, Windows, and Linux. Download, send a task - and boom, it just works with ZERO configuration! Download link in bio / first comment >>

Or Hiltch

103,543 Aufrufe • vor 5 Monaten

no money for grok or midjourney? this tool is for you. there's a FREE tool created by an anon dev. open-source. runs locally. 117k stars on github. it generates: > images & video > 3d models > audio > 20+ models here's how to set it up in under 5 minutes: 1️⃣download ComfyUI Desktop go to and grab the desktop app for your system. windows 10+, mac (apple silicon), or linux. it installs like any normal app, it sets up python and every dependency for you in the background. no terminal, no config files. 2️⃣open it first launch, it spins up its own environment automatically. you just wait a few seconds and you're in. you'll land on a node canvas, that's the whole interface. 3️⃣load a starter workflow top menu → Workflow → Browse Templates → Image Generation. click it. this drops a ready-made setup onto your canvas so you don't build anything from scratch. 4️⃣grab a model comfyui ships empty on purpose, the model is the brain, and you pick it. in the template, the "Load Checkpoint" node has a Download button when no model is installed. click it. it pulls one in for you (a few GB, this is the only real wait). 5️⃣install ComfyUI Manager this is the one add-on you don't skip. it lets you install models, custom nodes, and updates with a click instead of the command line. grab it from github (link in comments). it's the difference between fighting comfyui and flying in it. one honest note: an NVIDIA gpu makes this fast, apple silicon works great too, and a weak machine still runs it just slower. that's the whole setup. you now own an image, video, and 3D studio that costs you nothing per month. save this. and the next time grok or midjourney asks for your card. you won't need it. disclaimer: comfyui itself is 100% free. so are the local models (sdxl, flux, wan 2.2, ltx-2). some premium models like seedance are pay-per-use api models, only if you want top-tier quality. the free local ones cover most of what you need. (github link in the comments) follow and turn on post notification for daily AI contents.
1:13

no money for grok or midjourney? this tool is for you. there's a FREE tool created by an anon dev. open-source. runs locally. 117k stars on github. it generates: > images & video > 3d models > audio > 20+ models here's how to set it up in under 5 minutes: 1️⃣download ComfyUI Desktop go to and grab the desktop app for your system. windows 10+, mac (apple silicon), or linux. it installs like any normal app, it sets up python and every dependency for you in the background. no terminal, no config files. 2️⃣open it first launch, it spins up its own environment automatically. you just wait a few seconds and you're in. you'll land on a node canvas, that's the whole interface. 3️⃣load a starter workflow top menu → Workflow → Browse Templates → Image Generation. click it. this drops a ready-made setup onto your canvas so you don't build anything from scratch. 4️⃣grab a model comfyui ships empty on purpose, the model is the brain, and you pick it. in the template, the "Load Checkpoint" node has a Download button when no model is installed. click it. it pulls one in for you (a few GB, this is the only real wait). 5️⃣install ComfyUI Manager this is the one add-on you don't skip. it lets you install models, custom nodes, and updates with a click instead of the command line. grab it from github (link in comments). it's the difference between fighting comfyui and flying in it. one honest note: an NVIDIA gpu makes this fast, apple silicon works great too, and a weak machine still runs it just slower. that's the whole setup. you now own an image, video, and 3D studio that costs you nothing per month. save this. and the next time grok or midjourney asks for your card. you won't need it. disclaimer: comfyui itself is 100% free. so are the local models (sdxl, flux, wan 2.2, ltx-2). some premium models like seedance are pay-per-use api models, only if you want top-tier quality. the free local ones cover most of what you need. (github link in the comments) follow and turn on post notification for daily AI contents.

m0h

14,542 Aufrufe • vor 2 Monaten

🚀 OpenLLM 0.6 is out providing fast inference speed! Check out our video comparing #OpenLLM and #Ollama handling concurrent requests on the Llama 3 8B model. Ollama is ideal for local LLM deployment, but it is not designed for high concurrency scenarios essential for deployments in the cloud, powering applications with a large number of users. OpenLLM enables you to run any open-source model as OpenAI-compatible endpoints with just a single command. It supports a broad array of models like Llama 3, Mistral, and Qwen 2, including those fine-tuned with proprietary datasets and quantized versions. Ready to speed up your AI deployments? See the OpenLLM repo #BentoML #OpenSource
0:43

🚀 OpenLLM 0.6 is out providing fast inference speed! Check out our video comparing #OpenLLM and #Ollama handling concurrent requests on the Llama 3 8B model. Ollama is ideal for local LLM deployment, but it is not designed for high concurrency scenarios essential for deployments in the cloud, powering applications with a large number of users. OpenLLM enables you to run any open-source model as OpenAI-compatible endpoints with just a single command. It supports a broad array of models like Llama 3, Mistral, and Qwen 2, including those fine-tuned with proprietary datasets and quantized versions. Ready to speed up your AI deployments? See the OpenLLM repo #BentoML #OpenSource

BentoML

18,798 Aufrufe • vor 2 Jahren

Andrej Karpathy (co-founder of OpenAI, Eureka Labs): "The future of AI is not all in the cloud. The most important models will run locally on your hardware under your control." June 2026 is the first time that the future is actually accessible to everyone! Here is what the local LLM landscape looks like right now: → raspberry pi 5: coherent chatbot, no GPU required → macbook air: matches GPT-3.5 quality on most tasks natively → used RTX 3090 at $700: something close to GPT-4 running locally → ollama: one command to an openAI-compatible endpoint, every major toolchain integrates with it → LM studio: best GUI, native MLX acceleration on apple silicon, MCP support in 0.4.0 → llama.cpp: the engine everything else runs on, 90MB footprint, runs on raspberry pi to android → the honest case: 80% of daily work is genuinely good enough locally right now Your prompts never leave your machine. no per-token billing. no rate limits. no refusals. The playbook for every hardware tier, from Raspberry Pi to RTX 5090, is here. Bookmark it so you do not lose it! Follow NeilXbt for more local AI and open source intelligence that tracks the hardware and tooling most people discover six months late.
3:47

Andrej Karpathy (co-founder of OpenAI, Eureka Labs): "The future of AI is not all in the cloud. The most important models will run locally on your hardware under your control." June 2026 is the first time that the future is actually accessible to everyone! Here is what the local LLM landscape looks like right now: → raspberry pi 5: coherent chatbot, no GPU required → macbook air: matches GPT-3.5 quality on most tasks natively → used RTX 3090 at $700: something close to GPT-4 running locally → ollama: one command to an openAI-compatible endpoint, every major toolchain integrates with it → LM studio: best GUI, native MLX acceleration on apple silicon, MCP support in 0.4.0 → llama.cpp: the engine everything else runs on, 90MB footprint, runs on raspberry pi to android → the honest case: 80% of daily work is genuinely good enough locally right now Your prompts never leave your machine. no per-token billing. no rate limits. no refusals. The playbook for every hardware tier, from Raspberry Pi to RTX 5090, is here. Bookmark it so you do not lose it! Follow NeilXbt for more local AI and open source intelligence that tracks the hardware and tooling most people discover six months late.

NeilXbt

19,405 Aufrufe • vor 2 Monaten

A GIRL BOUGHT A $599 APPLE BOX AND CUT HER AI COSTS FROM $459/MONTH TO $23/MONTH. THE MAC MINI M4 IS QUIETLY BECOMING THE CHEAPEST AI SETUP IN 2026 she didn’t buy it because it looked nice on her desk. she bought it because paying for claude, chatgpt, cursor and api usage every month was getting ridiculous the setup is simple. mac mini m4, ollama, open webui, and local models like qwen, deepseek and llama. for most daily work, that’s enough to write, code, summarize, search notes, and run private workflows without sending everything to the cloud that’s why the math looks so good. a heavy ai stack can hit $459 a month, or $5,508 a year. the mac mini starts at $599 and uses around $3 a month in electricity. if it handles even 70 to 80% of the workload, it pays for itself fast install ollama, point your tools to localhost, and the workflow changes immediately. no token stress, no rate limits, no wondering where your files are going you still keep one cloud model for the hardest tasks but once a small box on your desk does most of the work, paying full price for everything starts to feel stupid
0:23

A GIRL BOUGHT A $599 APPLE BOX AND CUT HER AI COSTS FROM $459/MONTH TO $23/MONTH. THE MAC MINI M4 IS QUIETLY BECOMING THE CHEAPEST AI SETUP IN 2026 she didn’t buy it because it looked nice on her desk. she bought it because paying for claude, chatgpt, cursor and api usage every month was getting ridiculous the setup is simple. mac mini m4, ollama, open webui, and local models like qwen, deepseek and llama. for most daily work, that’s enough to write, code, summarize, search notes, and run private workflows without sending everything to the cloud that’s why the math looks so good. a heavy ai stack can hit $459 a month, or $5,508 a year. the mac mini starts at $599 and uses around $3 a month in electricity. if it handles even 70 to 80% of the workload, it pays for itself fast install ollama, point your tools to localhost, and the workflow changes immediately. no token stress, no rate limits, no wondering where your files are going you still keep one cloud model for the hardest tasks but once a small box on your desk does most of the work, paying full price for everything starts to feel stupid

Gipp 🦅

458,361 Aufrufe • vor 2 Monaten

🚨Stop gambling on the wrong local LLM. Most people download a model, watch it crawl, then start over. LLM Hardware Scanner is a CLI tool that scans your actual specs and ranks hundreds of LLMs before you waste the bandwidth: ⤷ Reads your RAM, CPU, and GPU to score real compatibility ⤷ Accounts for MoE activation not just bloated total parameter counts ⤷ Auto-selects the optimal quantization for your exact setup ⤷ Ranks models across quality, speed, context length, and true memory fit ⤷ Works natively with Ollama, llama.cpp, and MLX ⤷ Shows a composite score plus a speed estimate before you download a single byte Not a benchmark site. Not a spec sheet. A hardware-aware ranker that tells you the truth about what will actually run on your machine. One scan. Instant ranking. Zero wasted downloads.
0:21

🚨Stop gambling on the wrong local LLM. Most people download a model, watch it crawl, then start over. LLM Hardware Scanner is a CLI tool that scans your actual specs and ranks hundreds of LLMs before you waste the bandwidth: ⤷ Reads your RAM, CPU, and GPU to score real compatibility ⤷ Accounts for MoE activation not just bloated total parameter counts ⤷ Auto-selects the optimal quantization for your exact setup ⤷ Ranks models across quality, speed, context length, and true memory fit ⤷ Works natively with Ollama, llama.cpp, and MLX ⤷ Shows a composite score plus a speed estimate before you download a single byte Not a benchmark site. Not a spec sheet. A hardware-aware ranker that tells you the truth about what will actually run on your machine. One scan. Instant ranking. Zero wasted downloads.

Charlie Hills

37,499 Aufrufe • vor 5 Monaten

Why I use Cline - a free VS Code plugin - for AI engineering. Saoud Rizwan has: Flexible context management: It lets you include only what's relevant, making it ideal for large codebases. Attach files, folders, URLs and problems. Visualizes how much of the context window you've used too. Model flexibility: Cline isn't tied to one provider; it supports models from Anthropic, DeepSeek, Google Gemini, OpenAI, local models (via ollama or LM Studio) and more, allowing you to switch models for cost efficiency and capability. Plan/Act Modes: The v3.2.6 update introduces modes for designing solutions (Plan) or direct implementation (Act), providing control when you need it most. DeepSeek-R1 (Plan) + Claude 3.5 Sonnet (Act) workflow: This hybrid approach can reduce costs by up to 97% while improving output quality. DeepSeek as an architect, Sonnet for implementation. Checkpoints: Beyond git, Cline captures workspace states, offering granular rollback and comparison, especially useful for exploring solutions or debugging. Runtime awareness: Cline's integration with running systems is a game-changer, allowing real-time interaction with browsers (automatically check UI, interactions) and terminals to verify changes. Model Context Protocol (MCP): This allows for custom tool integration, making Cline extensible for specific enterprise needs without complex prompt engineering. How does it compare to alternatives? Cline shines with its system-level integration, model flexibility, and control, though it requires more attention to model selection and cost. My take: Cline aligns with professional engineering practices, offering control, visibility, and extensibility for complex system development. Totally worth considering for serious engineering teams.
0:55

Why I use Cline - a free VS Code plugin - for AI engineering. Saoud Rizwan has: Flexible context management: It lets you include only what's relevant, making it ideal for large codebases. Attach files, folders, URLs and problems. Visualizes how much of the context window you've used too. Model flexibility: Cline isn't tied to one provider; it supports models from Anthropic, DeepSeek, Google Gemini, OpenAI, local models (via ollama or LM Studio) and more, allowing you to switch models for cost efficiency and capability. Plan/Act Modes: The v3.2.6 update introduces modes for designing solutions (Plan) or direct implementation (Act), providing control when you need it most. DeepSeek-R1 (Plan) + Claude 3.5 Sonnet (Act) workflow: This hybrid approach can reduce costs by up to 97% while improving output quality. DeepSeek as an architect, Sonnet for implementation. Checkpoints: Beyond git, Cline captures workspace states, offering granular rollback and comparison, especially useful for exploring solutions or debugging. Runtime awareness: Cline's integration with running systems is a game-changer, allowing real-time interaction with browsers (automatically check UI, interactions) and terminals to verify changes. Model Context Protocol (MCP): This allows for custom tool integration, making Cline extensible for specific enterprise needs without complex prompt engineering. How does it compare to alternatives? Cline shines with its system-level integration, model flexibility, and control, though it requires more attention to model selection and cost. My take: Cline aligns with professional engineering practices, offering control, visibility, and extensibility for complex system development. Totally worth considering for serious engineering teams.

Addy Osmani

53,487 Aufrufe • vor 1 Jahr

Introducing WorldForge: testable world-model workflows for physical AI systems. You can think of it, loosely, as “LangChain for world models”. The problem is that “world model” has become an overloaded label. Depending on context, it can mean a video generator, a cost model, a robot policy, a JEPA-style latent predictor, etc. They share almost nothing, different inputs, runtimes, failure modes. I built WorldForge to stop pretending they're interchangeable. Front-door demo: a real Hugging Face LeRobot (LeRobot) diffusion_pusht policy combined with LeWorldModel by Lucas Maes checkpoint for scoring. Both run locally on my MacBook in the demo video. LeWM is extremely efficient (~15M params), can plan up to 48× faster, and runs on commodity hardware. WorldForge wires the loop: policy → candidates → score → select Replay happens in a local TUI today, but the same loop could drive a real robot in the world. Would love feedback from people working on world models, physical AI, robotics, ML infra, and adjacent tooling. Fully open source. Contributions very welcome. Plan in the dream, replay in real world.
16:27

Introducing WorldForge: testable world-model workflows for physical AI systems. You can think of it, loosely, as “LangChain for world models”. The problem is that “world model” has become an overloaded label. Depending on context, it can mean a video generator, a cost model, a robot policy, a JEPA-style latent predictor, etc. They share almost nothing, different inputs, runtimes, failure modes. I built WorldForge to stop pretending they're interchangeable. Front-door demo: a real Hugging Face LeRobot (LeRobot) diffusion_pusht policy combined with LeWorldModel by Lucas Maes checkpoint for scoring. Both run locally on my MacBook in the demo video. LeWM is extremely efficient (~15M params), can plan up to 48× faster, and runs on commodity hardware. WorldForge wires the loop: policy → candidates → score → select Replay happens in a local TUI today, but the same loop could drive a real robot in the world. Would love feedback from people working on world models, physical AI, robotics, ML infra, and adjacent tooling. Fully open source. Contributions very welcome. Plan in the dream, replay in real world.

abdel

17,001 Aufrufe • vor 3 Monaten