Загрузка видео...

Не удалось загрузить видео

На главную

Running Phi 4 Mini Instruct at 75 tok/sec locally on a Mac - powered by llama.cpp! 🔥 All you need is: brew install llama.cpp Followed by: llama-cli -hf bartowski/microsoft_Phi-4-mini-instruct-GGUF:Q8_0 That's it! 🤗

reach_vb's profile picture

Vaibhav (VB) Srivastav

32,039 subscribers

30,964 просмотров • 1 год назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

All thanks to @ggerganov for llama.cpp & @bartowski1182 for the quants:

Фото профиля glueckkanja
glueckkanja4 лет назад

Are you searching for a simple way to deploy device certificates with #Intune? Check out SCEPman at

Фото профиля luis
luis1 год назад

🙂‍↔️🤯

Фото профиля GawdOfAi
GawdOfAi1 год назад

why not use mlx?

Фото профиля Vaibhav (VB) Srivastav
Vaibhav (VB) Srivastav1 год назад

why not use both :)

Фото профиля Ankush Singal
Ankush Singal1 год назад

just added here:

Фото профиля Carlos (EUR/BCN)
Carlos (EUR/BCN)1 год назад

...and when for @ollama? Is it a different architecture?

Фото профиля Muhamad
Muhamad1 год назад

Can it handle audio input and output?

Фото профиля Hüseyin
Hüseyin1 год назад

Hello, which is the best model with decompilation capability for assembly c translation?

Фото профиля Anda
Anda1 год назад

That's some seriously smooth local inference speed—Phi 4 Mini's performance on llama.cpp is like watching a panda gracefully munch bamboo at hyper-speed! 🐼💨

Фото профиля bryan pratte
bryan pratte1 год назад

Have you tried SFT with audio/video with it?

Похожие видео

Fuck it, 685B parameter, DeepSeek V3 0324 running locally on M3 Ultra, fully private 🔥 Powered by llama.cpp & dynamic quants from Unsloth AI ⚡ Step 1: brew install llama.cpp Step 2: llama-cli -hf unsloth/DeepSeek-V3-0324-GGUF:Q2_K_XL That's it! 🤗 Honestly a bit surreal to be able to chat with such a chunky model at the touch of the keyboard - future is going to be wild!!
5:00

Fuck it, 685B parameter, DeepSeek V3 0324 running locally on M3 Ultra, fully private 🔥 Powered by llama.cpp & dynamic quants from Unsloth AI ⚡ Step 1: brew install llama.cpp Step 2: llama-cli -hf unsloth/DeepSeek-V3-0324-GGUF:Q2_K_XL That's it! 🤗 Honestly a bit surreal to be able to chat with such a chunky model at the touch of the keyboard - future is going to be wild!!

Vaibhav (VB) Srivastav

168,141 просмотров • 1 год назад

Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡
0:38

Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡

Vaibhav (VB) Srivastav

317,790 просмотров • 1 год назад

BOOM! Latest Qwen 30B A3B 2507 running blazingly fast on Mac powered by MLX 💥 mlx_lm.chat --model "lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit" That's it, try it out today!
1:38

BOOM! Latest Qwen 30B A3B 2507 running blazingly fast on Mac powered by MLX 💥 mlx_lm.chat --model "lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit" That's it, try it out today!

Vaibhav (VB) Srivastav

35,873 просмотров • 11 месяцев назад

Real-time webcam demo with Hugging Face SmolVLM and ggml llama.cpp server. All running locally on a Macbook M3
0:24

Real-time webcam demo with Hugging Face SmolVLM and ggml llama.cpp server. All running locally on a Macbook M3

Xuan-Son Nguyen

973,673 просмотров • 1 год назад

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)
0:59

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

783,815 просмотров • 3 месяцев назад

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:
0:59

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:

Andi Marafioti

76,150 просмотров • 1 месяц назад

In 44 lines of code, we have an actually useful agent that runs entirely locally, powered by Qwen3 30B A3B Instruct
1:22

In 44 lines of code, we have an actually useful agent that runs entirely locally, powered by Qwen3 30B A3B Instruct

Andrew I. Christianson

41,110 просмотров • 10 месяцев назад

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth
0:30

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

Lewis Tunstall

118,444 просмотров • 1 месяц назад

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓
0:24

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Pau Labarta Bajo

80,190 просмотров • 4 месяцев назад

Microsoft has released a new open source AI model Phi-4-mini is surprisingly powerful for its size (3.8B!) You can use it for free locally on your Windows, MacOS or Linux laptop. Links and tutorial below
0:39

Microsoft has released a new open source AI model Phi-4-mini is surprisingly powerful for its size (3.8B!) You can use it for free locally on your Windows, MacOS or Linux laptop. Links and tutorial below

Paul Couvert

209,124 просмотров • 1 год назад

This is literally the fastest way to install OpenClaw (MoltBot). This video will show you how to do it step by step. You don't need to buy a Mac Mini. We'll install it in DigitalOcean. I've installed this 12 different times already, and this is the fastest way I've found.
7:17

This is literally the fastest way to install OpenClaw (MoltBot). This video will show you how to do it step by step. You don't need to buy a Mac Mini. We'll install it in DigitalOcean. I've installed this 12 different times already, and this is the fastest way I've found.

Santiago

241,149 просмотров • 5 месяцев назад

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits​​​​​​​​​​​​​​​​ full breakdown of my raw llama.cpp setup ↓
21:03

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits​​​​​​​​​​​​​​​​ full breakdown of my raw llama.cpp setup ↓

leopardracer

238,950 просмотров • 1 месяц назад

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵
1:39

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

thestreamingdev()

727,259 просмотров • 3 месяцев назад

llama-4-scout-17b-16e-instruct prompt: write a p5.js script that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically
0:38

llama-4-scout-17b-16e-instruct prompt: write a p5.js script that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically

AK

118,360 просмотров • 1 год назад

Mirror: A hackable AI-powered Mirror on Your Laptop. Mirror is a simple yet powerful web app that runs 100% locally, where the AI (Georgi Gerganov's llama.cpp + skunkworks's Bakllava) constantly watches your webcam feed and sends you messages. And you can try it too, right now.
1:21

Mirror: A hackable AI-powered Mirror on Your Laptop. Mirror is a simple yet powerful web app that runs 100% locally, where the AI (Georgi Gerganov's llama.cpp + skunkworks's Bakllava) constantly watches your webcam feed and sends you messages. And you can try it too, right now.

cocktail peanut

140,263 просмотров • 2 лет назад

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡
0:44

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡

Vaibhav (VB) Srivastav

51,389 просмотров • 1 год назад