Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Running Phi 4 Mini Instruct at 75 tok/sec locally on a Mac - powered by llama.cpp! 🔥 All you need is: brew install llama.cpp Followed by: llama-cli -hf bartowski/microsoft_Phi-4-mini-instruct-GGUF:Q8_0 That's it! 🤗

reach_vb's profile picture

Vaibhav (VB) Srivastav

32,039 subscribers

30,964 Aufrufe • vor 1 Jahr •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastavvor 1 Jahr

All thanks to @ggerganov for llama.cpp & @bartowski1182 for the quants:

Profilbild von glueckkanja
glueckkanjavor 4 Jahren

Are you searching for a simple way to deploy device certificates with #Intune? Check out SCEPman at

Profilbild von luis
luisvor 1 Jahr

🙂‍↔️🤯

Profilbild von GawdOfAi
GawdOfAivor 1 Jahr

why not use mlx?

Profilbild von Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastavvor 1 Jahr

why not use both :)

Profilbild von Ankush Singal
Ankush Singalvor 1 Jahr

just added here:

Profilbild von Carlos (EUR/BCN)
Carlos (EUR/BCN)vor 1 Jahr

...and when for @ollama? Is it a different architecture?

Profilbild von Muhamad
Muhamadvor 1 Jahr

Can it handle audio input and output?

Profilbild von Hüseyin
Hüseyinvor 1 Jahr

Hello, which is the best model with decompilation capability for assembly c translation?

Profilbild von Anda
Andavor 1 Jahr

That's some seriously smooth local inference speed—Phi 4 Mini's performance on llama.cpp is like watching a panda gracefully munch bamboo at hyper-speed! 🐼💨

Profilbild von bryan pratte
bryan prattevor 1 Jahr

Have you tried SFT with audio/video with it?

Ähnliche Videos

Fuck it, 685B parameter, DeepSeek V3 0324 running locally on M3 Ultra, fully private 🔥 Powered by llama.cpp & dynamic quants from Unsloth AI ⚡ Step 1: brew install llama.cpp Step 2: llama-cli -hf unsloth/DeepSeek-V3-0324-GGUF:Q2_K_XL That's it! 🤗 Honestly a bit surreal to be able to chat with such a chunky model at the touch of the keyboard - future is going to be wild!!
5:00

Fuck it, 685B parameter, DeepSeek V3 0324 running locally on M3 Ultra, fully private 🔥 Powered by llama.cpp & dynamic quants from Unsloth AI ⚡ Step 1: brew install llama.cpp Step 2: llama-cli -hf unsloth/DeepSeek-V3-0324-GGUF:Q2_K_XL That's it! 🤗 Honestly a bit surreal to be able to chat with such a chunky model at the touch of the keyboard - future is going to be wild!!

Vaibhav (VB) Srivastav

168,141 Aufrufe • vor 1 Jahr

Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡
0:38

Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡

Vaibhav (VB) Srivastav

317,790 Aufrufe • vor 1 Jahr

BOOM! Latest Qwen 30B A3B 2507 running blazingly fast on Mac powered by MLX 💥 mlx_lm.chat --model "lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit" That's it, try it out today!
1:38

BOOM! Latest Qwen 30B A3B 2507 running blazingly fast on Mac powered by MLX 💥 mlx_lm.chat --model "lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit" That's it, try it out today!

Vaibhav (VB) Srivastav

35,873 Aufrufe • vor 11 Monaten

Real-time webcam demo with Hugging Face SmolVLM and ggml llama.cpp server. All running locally on a Macbook M3
0:24

Real-time webcam demo with Hugging Face SmolVLM and ggml llama.cpp server. All running locally on a Macbook M3

Xuan-Son Nguyen

973,673 Aufrufe • vor 1 Jahr

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)
0:59

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

783,815 Aufrufe • vor 3 Monaten

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:
0:59

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:

Andi Marafioti

76,150 Aufrufe • vor 1 Monat

In 44 lines of code, we have an actually useful agent that runs entirely locally, powered by Qwen3 30B A3B Instruct
1:22

In 44 lines of code, we have an actually useful agent that runs entirely locally, powered by Qwen3 30B A3B Instruct

Andrew I. Christianson

41,110 Aufrufe • vor 10 Monaten

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth
0:30

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

Lewis Tunstall

118,444 Aufrufe • vor 1 Monat

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓
0:24

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Pau Labarta Bajo

80,190 Aufrufe • vor 4 Monaten

Microsoft has released a new open source AI model Phi-4-mini is surprisingly powerful for its size (3.8B!) You can use it for free locally on your Windows, MacOS or Linux laptop. Links and tutorial below
0:39

Microsoft has released a new open source AI model Phi-4-mini is surprisingly powerful for its size (3.8B!) You can use it for free locally on your Windows, MacOS or Linux laptop. Links and tutorial below

Paul Couvert

209,124 Aufrufe • vor 1 Jahr

This is literally the fastest way to install OpenClaw (MoltBot). This video will show you how to do it step by step. You don't need to buy a Mac Mini. We'll install it in DigitalOcean. I've installed this 12 different times already, and this is the fastest way I've found.
7:17

This is literally the fastest way to install OpenClaw (MoltBot). This video will show you how to do it step by step. You don't need to buy a Mac Mini. We'll install it in DigitalOcean. I've installed this 12 different times already, and this is the fastest way I've found.

Santiago

241,149 Aufrufe • vor 5 Monaten

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits​​​​​​​​​​​​​​​​ full breakdown of my raw llama.cpp setup ↓
21:03

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits​​​​​​​​​​​​​​​​ full breakdown of my raw llama.cpp setup ↓

leopardracer

238,950 Aufrufe • vor 1 Monat

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵
1:39

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

thestreamingdev()

727,259 Aufrufe • vor 3 Monaten

llama-4-scout-17b-16e-instruct prompt: write a p5.js script that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically
0:38

llama-4-scout-17b-16e-instruct prompt: write a p5.js script that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically

AK

118,360 Aufrufe • vor 1 Jahr

Mirror: A hackable AI-powered Mirror on Your Laptop. Mirror is a simple yet powerful web app that runs 100% locally, where the AI (Georgi Gerganov's llama.cpp + skunkworks's Bakllava) constantly watches your webcam feed and sends you messages. And you can try it too, right now.
1:21

Mirror: A hackable AI-powered Mirror on Your Laptop. Mirror is a simple yet powerful web app that runs 100% locally, where the AI (Georgi Gerganov's llama.cpp + skunkworks's Bakllava) constantly watches your webcam feed and sends you messages. And you can try it too, right now.

cocktail peanut

140,263 Aufrufe • vor 2 Jahren

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡
0:44

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡

Vaibhav (VB) Srivastav

51,389 Aufrufe • vor 1 Jahr