Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

STOP GUESSING WHICH LOCAL AI MODELS YOUR COMPUTER CAN RUN. LLMFIT SCANS YOUR HARDWARE AND TELLS YOU WHICH MODELS WILL ACTUALLY RUN WELL, INCLUDING THE BEST QUANTIZATION TO USE.

0xMarioNawfal

263,020 subscribers

48,706 views • 13 days ago •via X (Twitter)

News & Politics Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

llmfit. Useful tool that probes hardware and tells you exactly which LLMs will actually run. - handles MoE expert offloading, picks the best quantization for your RAM, estimates tokens/sec before you even pull the weights. Essential for local dev.

llmfit. Useful tool that probes hardware and tells you exactly which LLMs will actually run. - handles MoE expert offloading, picks the best quantization for your RAM, estimates tokens/sec before you even pull the weights. Essential for local dev.

Wildminder

248,651 views • 7 months ago

Tired of guessing which AI model is best? 🤔 We've got you covered! 👀 Here's a sneak peek of our new model comparison feature in action! Run the same prompt across multiple models and see which one works best for YOU. Which models are you excited to test? #Inferium #AI #Inference #Compare

Tired of guessing which AI model is best? 🤔 We've got you covered! 👀 Here's a sneak peek of our new model comparison feature in action! Run the same prompt across multiple models and see which one works best for YOU. Which models are you excited to test? #Inferium #AI #Inference #Compare

Inferium

12,352 views • 1 year ago

Fable 5 was banned by the US government yesterday. It's time to build your own Personal AI Computer and run local models. So no one can ever cut you off. Here's how ↓

Fable 5 was banned by the US government yesterday. It's time to build your own Personal AI Computer and run local models. So no one can ever cut you off. Here's how ↓

Dee

309,938 views • 3 months ago

Stop using frontier models for your entire OpenClaw stack. It's expensive and wastes your quota. The first step is identifying which use cases CAN be moved to local models. Here's how I do that:

Stop using frontier models for your entire OpenClaw stack. It's expensive and wastes your quota. The first step is identifying which use cases CAN be moved to local models. Here's how I do that:

Matthew Berman

58,095 views • 5 months ago

The #1 problem with local AI is now solved. There’s a free tool called that checks your hardware and tells you which models will actually run well before you download anything. So instead of guessing and hitting out-of-memory errors…it grades every model against your machine. What it does (right in your browser, no install): → detects your setup (RAM / CPU / GPU / VRAM) → scores each model for fit, speed, and context length → grades every quantization level (Q4_K_M, Q6_K, Q8_0, etc.) → labels what runs great vs okay vs too heavy It covers most of the open-weight stack, Llama, Qwen, Gemma, Mistral, DeepSeek, Phi and more, pulling requirements from llama.cpp, Ollama, and LM Studio. And it’s fully opensource

The #1 problem with local AI is now solved. There’s a free tool called that checks your hardware and tells you which models will actually run well before you download anything. So instead of guessing and hitting out-of-memory errors…it grades every model against your machine. What it does (right in your browser, no install): → detects your setup (RAM / CPU / GPU / VRAM) → scores each model for fit, speed, and context length → grades every quantization level (Q4_K_M, Q6_K, Q8_0, etc.) → labels what runs great vs okay vs too heavy It covers most of the open-weight stack, Llama, Qwen, Gemma, Mistral, DeepSeek, Phi and more, pulling requirements from llama.cpp, Ollama, and LM Studio. And it’s fully opensource

Alvaro Cintas

193,286 views • 1 month ago

LLMs can take gigabytes of memory to store, which limits what can be run on consumer hardware. But quantization can dramatically compress models, making a wider selection of models available to developers. You can often reduce model size by 4x or more while maintaining reasonable performance. In our new short course Quantization Fundamentals taught by Hugging Face's @younesbelkada and Marc Sun, you'll: - Learn how to quantize nearly any open source model - Use int8 and bfloat16 (Brain float 16) data types to load and run LLMs using PyTorch and the Hugging Face Transformers library - Dive into the technical details of linear quantization to map 32-bit floats to 8-bit integers As models get bigger and bigger, quantization becomes more important for making models practical and accessible. Please check out the course here:

LLMs can take gigabytes of memory to store, which limits what can be run on consumer hardware. But quantization can dramatically compress models, making a wider selection of models available to developers. You can often reduce model size by 4x or more while maintaining reasonable performance. In our new short course Quantization Fundamentals taught by Hugging Face's @younesbelkada and Marc Sun, you'll: - Learn how to quantize nearly any open source model - Use int8 and bfloat16 (Brain float 16) data types to load and run LLMs using PyTorch and the Hugging Face Transformers library - Dive into the technical details of linear quantization to map 32-bit floats to 8-bit integers As models get bigger and bigger, quantization becomes more important for making models practical and accessible. Please check out the course here:

Andrew Ng

288,354 views • 2 years ago

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below

Paul Couvert

41,245 views • 6 months ago

Which models can your machine actually run? Magnitude is a 100% free, open source desktop app that: - profiles your hardware and runs sample calculations - predicts tok/s for every model before you download - recommends the best models, from fast to smart Pick your models and it handles the rest: - downloads and tunes the models for your hardware - connects agents like pi, opencode, hermes in one click - runs models on demand as your agent works Works on whatever hardware you already own: MacBook (Intel or Apple Silicon), Mac Mini, Mac Studio, DGX Spark, Strix Halo, any NVIDIA/AMD GPU, or just a CPU Download it on macOS, Windows, or Linux: Open source:

Which models can your machine actually run? Magnitude is a 100% free, open source desktop app that: - profiles your hardware and runs sample calculations - predicts tok/s for every model before you download - recommends the best models, from fast to smart Pick your models and it handles the rest: - downloads and tunes the models for your hardware - connects agents like pi, opencode, hermes in one click - runs models on demand as your agent works Works on whatever hardware you already own: MacBook (Intel or Apple Silicon), Mac Mini, Mac Studio, DGX Spark, Strix Halo, any NVIDIA/AMD GPU, or just a CPU Download it on macOS, Windows, or Linux: Open source:

Tom Greenwald

47,420 views • 5 days ago

Introducing Magnitude: your actually local agent 100% private and offline. No token costs, no API keys. Open source. Today's agents are local. The model isn't. Every prompt, every file, every secret gets sent straight to Anthropic and OpenAI. Magnitude is built around local models and runs the whole stack itself. The inference engine is part of the agent, so the models run inside it, right on your computer. It lives in your terminal, and setup is one command. Magnitude profiles your hardware and shows you which models fit, with the trade-offs between quality, speed, and memory. Pick one and start working. No painful config or server to babysit. Out of the box, it can use your shell, edit files, and run scripts. Add skills and it can work with Excel, PowerPoint, PDFs or Chrome. Use it for everyday work: - Analyze sensitive data - Manage private notes - Review code and logs - Search and organize files - Build docs or slides npm i -g @magnitudedev/cli GitHub:

Introducing Magnitude: your actually local agent 100% private and offline. No token costs, no API keys. Open source. Today's agents are local. The model isn't. Every prompt, every file, every secret gets sent straight to Anthropic and OpenAI. Magnitude is built around local models and runs the whole stack itself. The inference engine is part of the agent, so the models run inside it, right on your computer. It lives in your terminal, and setup is one command. Magnitude profiles your hardware and shows you which models fit, with the trade-offs between quality, speed, and memory. Pick one and start working. No painful config or server to babysit. Out of the box, it can use your shell, edit files, and run scripts. Add skills and it can work with Excel, PowerPoint, PDFs or Chrome. Use it for everyday work: - Analyze sensitive data - Manage private notes - Review code and logs - Search and organize files - Build docs or slides npm i -g @magnitudedev/cli GitHub:

Tom Greenwald

396,947 views • 1 month ago

Anthropic doesn't want you running local AI Ahmad explains why the labs are lying to you and how to run your own models instead

Anthropic doesn't want you running local AI Ahmad explains why the labs are lying to you and how to run your own models instead

David Ondrej

67,069 views • 27 days ago

The value produced by models is getting so much better so fast that old hardware is actually getting *more* expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

The value produced by models is getting so much better so fast that old hardware is actually getting more expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

Dwarkesh Patel

78,277 views • 6 months ago

The easiest way to run your agent harnesses using local models: Whether you're using Claude Code, Codex, OpenCode, or Pi, the local setup itself isn't really the hard part. The harder problem is figuring out which local model your machine can actually handle well. You have to think about RAM, model size, quantization, context length, KV cache, speed, accuracy, and a bunch of other trade-offs before you even start. Magnitude is an open-source solution that removes all of that guesswork for you. It profiles your machine, benchmarks what it can realistically run, recommends the best models for your hardware, and then lets you connect them to your preferred agent harness. The entire setup takes just two commands. In this video, I walk through the full process from hardware profiling to running a harness on a local model. Chapters: 00:00 Intro 00:17 What Magnitude is 00:45 Setup in two commands 01:29 Why picking a local model gets confusing fast 02:11 Hardware profiling + model recommendations 02:59 Choosing your harness 03:22 Connecting a harness that's already running 04:18 Outro Get started: (don't forget to star 🌟)

The easiest way to run your agent harnesses using local models: Whether you're using Claude Code, Codex, OpenCode, or Pi, the local setup itself isn't really the hard part. The harder problem is figuring out which local model your machine can actually handle well. You have to think about RAM, model size, quantization, context length, KV cache, speed, accuracy, and a bunch of other trade-offs before you even start. Magnitude is an open-source solution that removes all of that guesswork for you. It profiles your machine, benchmarks what it can realistically run, recommends the best models for your hardware, and then lets you connect them to your preferred agent harness. The entire setup takes just two commands. In this video, I walk through the full process from hardware profiling to running a harness on a local model. Chapters: 00:00 Intro 00:17 What Magnitude is 00:45 Setup in two commands 01:29 Why picking a local model gets confusing fast 02:11 Hardware profiling + model recommendations 02:59 Choosing your harness 03:22 Connecting a harness that's already running 04:18 Outro Get started: (don't forget to star 🌟)

Akshay 🚀

18,722 views • 8 days ago

Portable Computer is now available on Windows PCs with @NVIDIA RTX GPUs. Run the harness, agents, and models locally on your PC. Work with local files and connected apps without sending tasks to the cloud. Use frontier cloud models when needed.

Portable Computer is now available on Windows PCs with @NVIDIA RTX GPUs. Run the harness, agents, and models locally on your PC. Work with local files and connected apps without sending tasks to the cloud. Use frontier cloud models when needed.

Perplexity

634,917 views • 9 days ago

WOW! OPENAI DID IT! They released their first TWO open weights models! This is HUGE news for the entire AI industry. It means you can RIGHT NOW run cutting edge AI models locally on your computer Here's all the benefits and how to set up the models locally yourself:

WOW! OPENAI DID IT! They released their first TWO open weights models! This is HUGE news for the entire AI industry. It means you can RIGHT NOW run cutting edge AI models locally on your computer Here's all the benefits and how to set up the models locally yourself:

Alex Finn

71,875 views • 1 year ago

I always wondered which Ollama model my machine could run properly. Now you can easily do that using llms-checker It's a CLI tool that checks your system specs and recommends the best models to run on Ollama based on what your machine npm install -g llm-checker

I always wondered which Ollama model my machine could run properly. Now you can easily do that using llms-checker It's a CLI tool that checks your system specs and recommends the best models to run on Ollama based on what your machine npm install -g llm-checker

Jaydeep

112,465 views • 7 months ago

Run Cline on Local AI models with Atomic Chat! Cline is a coding agent trusted by 8M+ developers. Write, refactor, ship code securely on your own hardware with local models powered by atomic.chat — no cloud, private, free and open-source

Run Cline on Local AI models with Atomic Chat! Cline is a coding agent trusted by 8M+ developers. Write, refactor, ship code securely on your own hardware with local models powered by atomic.chat — no cloud, private, free and open-source

atomic.chat

44,433 views • 3 months ago

🧳 Bring Your Own AI Models to GitHub Copilot in VS Code Use your own Azure-hosted AI models with GitHub Copilot in VS Code. See how Bring Your Own Key (BYOK) lets you connect models from Microsoft Foundry, choose which models appear in Copilot, and use the right model for different coding tasks.

🧳 Bring Your Own AI Models to GitHub Copilot in VS Code Use your own Azure-hosted AI models with GitHub Copilot in VS Code. See how Bring Your Own Key (BYOK) lets you connect models from Microsoft Foundry, choose which models appear in Copilot, and use the right model for different coding tasks.

Visual Studio Code

38,295 views • 12 days ago

Introducing Vals-Smith: turn your code base into a customized benchmark. Public benchmarks tell you which model is strongest overall, not which model is the best on your code. Vals-Smith turns your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. New models ship every week. Vals-Smith tells you which one to trust with your code.

Introducing Vals-Smith: turn your code base into a customized benchmark. Public benchmarks tell you which model is strongest overall, not which model is the best on your code. Vals-Smith turns your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. New models ship every week. Vals-Smith tells you which one to trust with your code.

Vals AI

46,485 views • 2 months ago

Fable is banned. Long live local AI. Full episode breaking down exactly how to get good at local models. the runtime, the hardware, quantization, connecting it to Hermes agent and local AI startup ideas (25 minutes)

Fable is banned. Long live local AI. Full episode breaking down exactly how to get good at local models. the runtime, the hardware, quantization, connecting it to Hermes agent and local AI startup ideas (25 minutes)

GREG ISENBERG

158,205 views • 3 months ago

Do you see what's happening? Local AI is getting smarter. It's Opus 4.5 level Hardware that hosts those models are flying off the shelf Your window to host local models is shrinking In this video I cover EVERYTHING local AI models and how to host your own, even on Mac Minis:

Do you see what's happening? Local AI is getting smarter. It's Opus 4.5 level Hardware that hosts those models are flying off the shelf Your window to host local models is shrinking In this video I cover EVERYTHING local AI models and how to host your own, even on Mac Minis:

Alex Finn

55,389 views • 6 months ago