Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Running Phi 4 Mini Instruct at 75 tok/sec locally on a Mac - powered by llama.cpp! 🔥 All you need is: brew install llama.cpp Followed by: llama-cli -hf bartowski/microsoft_Phi-4-mini-instruct-GGUF:Q8_0 That's it! 🤗

Vaibhav (VB) Srivastav

32,039 subscribers

30,964 görüntüleme • 1 yıl önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

11 Yorum

Vaibhav (VB) Srivastav profil fotoğrafı

Vaibhav (VB) Srivastav1 yıl önce

All thanks to @ggerganov for llama.cpp & @bartowski1182 for the quants:

glueckkanja profil fotoğrafı

glueckkanja4 yıl önce

Are you searching for a simple way to deploy device certificates with #Intune? Check out SCEPman at

luis profil fotoğrafı

luis1 yıl önce

🙂‍↔️🤯

GawdOfAi profil fotoğrafı

GawdOfAi1 yıl önce

why not use mlx?

Vaibhav (VB) Srivastav profil fotoğrafı

Vaibhav (VB) Srivastav1 yıl önce

why not use both :)

Ankush Singal profil fotoğrafı

Ankush Singal1 yıl önce

just added here:

Carlos (EUR/BCN) profil fotoğrafı

Carlos (EUR/BCN)1 yıl önce

...and when for @ollama? Is it a different architecture?

Muhamad profil fotoğrafı

Muhamad1 yıl önce

Can it handle audio input and output?

Hüseyin profil fotoğrafı

Hüseyin1 yıl önce

Hello, which is the best model with decompilation capability for assembly c translation?

Anda profil fotoğrafı

Anda1 yıl önce

That's some seriously smooth local inference speed—Phi 4 Mini's performance on llama.cpp is like watching a panda gracefully munch bamboo at hyper-speed! 🐼💨

bryan pratte profil fotoğrafı

bryan pratte1 yıl önce

Have you tried SFT with audio/video with it?

Benzer Videolar

Fuck it, 685B parameter, DeepSeek V3 0324 running locally on M3 Ultra, fully private 🔥 Powered by llama.cpp & dynamic quants from Unsloth AI ⚡ Step 1: brew install llama.cpp Step 2: llama-cli -hf unsloth/DeepSeek-V3-0324-GGUF:Q2_K_XL That's it! 🤗 Honestly a bit surreal to be able to chat with such a chunky model at the touch of the keyboard - future is going to be wild!!

Fuck it, 685B parameter, DeepSeek V3 0324 running locally on M3 Ultra, fully private 🔥 Powered by llama.cpp & dynamic quants from Unsloth AI ⚡ Step 1: brew install llama.cpp Step 2: llama-cli -hf unsloth/DeepSeek-V3-0324-GGUF:Q2_K_XL That's it! 🤗 Honestly a bit surreal to be able to chat with such a chunky model at the touch of the keyboard - future is going to be wild!!

Vaibhav (VB) Srivastav

168,141 görüntüleme • 1 yıl önce

$Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡$

Fuck it! You can now run any GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡

Vaibhav (VB) Srivastav

317,790 görüntüleme • 1 yıl önce

BOOM! Latest Qwen 30B A3B 2507 running blazingly fast on Mac powered by MLX 💥 mlx_lm.chat --model "lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit" That's it, try it out today!

BOOM! Latest Qwen 30B A3B 2507 running blazingly fast on Mac powered by MLX 💥 mlx_lm.chat --model "lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit" That's it, try it out today!

Vaibhav (VB) Srivastav

35,873 görüntüleme • 11 ay önce

Real-time webcam demo with Hugging Face SmolVLM and ggml llama.cpp server. All running locally on a Macbook M3

Real-time webcam demo with Hugging Face SmolVLM and ggml llama.cpp server. All running locally on a Macbook M3

Xuan-Son Nguyen

973,673 görüntüleme • 1 yıl önce

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

783,815 görüntüleme • 3 ay önce

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:

Andi Marafioti

76,150 görüntüleme • 1 ay önce

In 44 lines of code, we have an actually useful agent that runs entirely locally, powered by Qwen3 30B A3B Instruct

In 44 lines of code, we have an actually useful agent that runs entirely locally, powered by Qwen3 30B A3B Instruct

Andrew I. Christianson

41,110 görüntüleme • 10 ay önce

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

Lewis Tunstall

118,444 görüntüleme • 1 ay önce

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Pau Labarta Bajo

80,190 görüntüleme • 4 ay önce

Microsoft has released a new open source AI model Phi-4-mini is surprisingly powerful for its size (3.8B!) You can use it for free locally on your Windows, MacOS or Linux laptop. Links and tutorial below

Microsoft has released a new open source AI model Phi-4-mini is surprisingly powerful for its size (3.8B!) You can use it for free locally on your Windows, MacOS or Linux laptop. Links and tutorial below

Paul Couvert

209,124 görüntüleme • 1 yıl önce

This is literally the fastest way to install OpenClaw (MoltBot). This video will show you how to do it step by step. You don't need to buy a Mac Mini. We'll install it in DigitalOcean. I've installed this 12 different times already, and this is the fastest way I've found.

This is literally the fastest way to install OpenClaw (MoltBot). This video will show you how to do it step by step. You don't need to buy a Mac Mini. We'll install it in DigitalOcean. I've installed this 12 different times already, and this is the fastest way I've found.

Santiago

241,149 görüntüleme • 5 ay önce

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits full breakdown of my raw llama.cpp setup ↓

THIS AMERICAN DEVELOPER SPENT WEEKS DEBUGGING TIMEOUT ERRORS IN OLLAMA. THEN HE LOOKED UNDER THE HOOD LM Studio is just llama.cpp Ollama is just llama.cpp so he cloned llama.cpp from source, pulled Qwen 3.6 35B off Hugging Face, set up asymmetric KV quantization and got a local server running on 127.0.0.1:8080 plugged it into VS Code, connected it to OpenClaw, 53 tok/s on an M1 Max with 262K context zero wrappers, zero timeout errors, zero API fees bookmark & like this before your next timeout error hits full breakdown of my raw llama.cpp setup ↓

leopardracer

238,950 görüntüleme • 1 ay önce

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

thestreamingdev()

727,259 görüntüleme • 3 ay önce

llama-4-scout-17b-16e-instruct prompt: write a p5.js script that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically

llama-4-scout-17b-16e-instruct prompt: write a p5.js script that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically

AK

118,360 görüntüleme • 1 yıl önce

Mirror: A hackable AI-powered Mirror on Your Laptop. Mirror is a simple yet powerful web app that runs 100% locally, where the AI (Georgi Gerganov's llama.cpp + skunkworks's Bakllava) constantly watches your webcam feed and sends you messages. And you can try it too, right now.

Mirror: A hackable AI-powered Mirror on Your Laptop. Mirror is a simple yet powerful web app that runs 100% locally, where the AI (Georgi Gerganov's llama.cpp + skunkworks's Bakllava) constantly watches your webcam feed and sends you messages. And you can try it too, right now.

cocktail peanut

140,263 görüntüleme • 2 yıl önce

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡

Vaibhav (VB) Srivastav

51,389 görüntüleme • 1 yıl önce