Georgi Gerganov's banner

Georgi Gerganov

@ggerganov • 71,455 subscribers

24th at the Electrica puzzle challenge | building https://t.co/baTQS2bL7I | engineer @huggingface

Shorts

HuggingFace just shipped in-browser GGUF editing It allows you to edit GGUF metadata in the comfort of your browser, without having to even download the full model. This feature is enabled via the Xet technology that makes partial file updates possible.

HuggingFace just shipped in-browser GGUF editing It allows you to edit GGUF metadata in the comfort of your browser, without having to even download the full model. This feature is enabled via the Xet technology that makes partial file updates possible.

37,314 görüntüleme

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

785,456 görüntüleme • 5 ay önce

Pro tip - hook your PC and Phone with Tailscale and enjoy fast and private inference on the go. Here is Gemma 4, hosted on Mac Studio, streaming to my iPhone. No 3rd party apps. Same WebUI experience everywhere.

Pro tip - hook your PC and Phone with Tailscale and enjoy fast and private inference on the go. Here is Gemma 4, hosted on Mac Studio, streaming to my iPhone. No 3rd party apps. Same WebUI experience everywhere.

Georgi Gerganov

371,360 görüntüleme • 5 ay önce

The Llama app for Mac now comes with a simple request builder for llama.cpp's REST API

The Llama app for Mac now comes with a simple request builder for llama.cpp's REST API

Georgi Gerganov

19,544 görüntüleme • 7 gün önce

Introducing LLaMA voice chat! 🦙 You can run this locally on an M1 Pro

Introducing LLaMA voice chat! 🦙 You can run this locally on an M1 Pro

Georgi Gerganov

1,682,130 görüntüleme • 3 yıl önce

Full F16 precision 34B Code Llama at >20 t/s on M2 Ultra

Full F16 precision 34B Code Llama at >20 t/s on M2 Ultra

Georgi Gerganov

1,158,151 görüntüleme • 3 yıl önce

Casually running a 180B parameter LLM on M2 Ultra

Casually running a 180B parameter LLM on M2 Ultra

Georgi Gerganov

683,236 görüntüleme • 3 yıl önce

Here is another Gibberlink experiment: Two AI agents autonomously encrypt their audio chat (video by Anton Pidkuiko)

Here is another Gibberlink experiment: Two AI agents autonomously encrypt their audio chat (video by Anton Pidkuiko)

Georgi Gerganov

270,537 görüntüleme • 1 yıl önce

DeepSeek-R1 on Mac Studio 192GB 🪄

DeepSeek-R1 on Mac Studio 192GB 🪄

Georgi Gerganov

259,545 görüntüleme • 1 yıl önce

LLaMA voice chat + Siri TTS This example is now truly 100% offline since we are now using the built-in Siri text-to-speech available on MacOS through the "say" command

LLaMA voice chat + Siri TTS This example is now truly 100% offline since we are now using the built-in Siri text-to-speech available on MacOS through the "say" command

Georgi Gerganov

412,007 görüntüleme • 3 yıl önce

Causally running Grok-1 at home

Causally running Grok-1 at home

Georgi Gerganov

276,954 görüntüleme • 2 yıl önce

sam.cpp 👀 Inference of Meta's Segment Anything Model on the CPU Project by Yavor Ivanov - powered by

sam.cpp 👀 Inference of Meta's Segment Anything Model on the CPU Project by Yavor Ivanov - powered by

Georgi Gerganov

264,664 görüntüleme • 3 yıl önce

Here is how to deploy and serve any LLM on HF with a single command in less than 3 minutes with llama.cpp $ bash -c "$(curl -s

Here is how to deploy and serve any LLM on HF with a single command in less than 3 minutes with llama.cpp $ bash -c "$(curl -s

Georgi Gerganov

143,145 görüntüleme • 2 yıl önce

The example below is using prompt-based speculative decoding. Specifically, ngram hashing is utilized to suggest drafts of up to 64 tokens. The hasher keeps track of ngrams in the observed contexts, so mostly effective for coding tasks. Here is another demo:

The example below is using prompt-based speculative decoding. Specifically, ngram hashing is utilized to suggest drafts of up to 64 tokens. The hasher keeps track of ngrams in the observed contexts, so mostly effective for coding tasks. Here is another demo:

Georgi Gerganov

29,890 görüntüleme • 5 ay önce

llama.vscode (powered by Qwen Coder)

llama.vscode (powered by Qwen Coder)

Georgi Gerganov

77,668 görüntüleme • 1 yıl önce

GGUF My Repo by Hugging Face Create quantum GGUF models fully online - quickly and secure. Thanks to Vaibhav (VB) Srivastav, Pedro Cuenca and team for creating this HF space! In the video below I give it a try to create a quantum 8-bit model of Gemma 2B - it took about 60 seconds. The resulting model becomes automatically available in your HF profile and is ready to be used with llama.cpp

Georgi Gerganov

64,511 görüntüleme • 2 yıl önce

Updated the llama.vim plugin to support speculative FIM It now generates the next suggestion while you review the current one. If you accept it, the next suggestion is available immediately.

Updated the llama.vim plugin to support speculative FIM It now generates the next suggestion while you review the current one. If you accept it, the next suggestion is available immediately.

Georgi Gerganov

28,337 görüntüleme • 1 yıl önce