Georgi Gerganov's banner

Georgi Gerganov

@ggerganov • 73,223 subscribers

24th at the Electrica puzzle challenge | building https://t.co/baTQS2bL7I | engineer @huggingface

Shorts

HuggingFace just shipped in-browser GGUF editing It allows you to edit GGUF metadata in the comfort of your browser, without having to even download the full model. This feature is enabled via the Xet technology that makes partial file updates possible.

HuggingFace just shipped in-browser GGUF editing It allows you to edit GGUF metadata in the comfort of your browser, without having to even download the full model. This feature is enabled via the Xet technology that makes partial file updates possible.

37,314 次观看

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

786,103 次观看 • 6 个月前

Pro tip - hook your PC and Phone with Tailscale and enjoy fast and private inference on the go. Here is Gemma 4, hosted on Mac Studio, streaming to my iPhone. No 3rd party apps. Same WebUI experience everywhere.

Pro tip - hook your PC and Phone with Tailscale and enjoy fast and private inference on the go. Here is Gemma 4, hosted on Mac Studio, streaming to my iPhone. No 3rd party apps. Same WebUI experience everywhere.

Georgi Gerganov

372,136 次观看 • 6 个月前

Introducing LLaMA voice chat! 🦙 You can run this locally on an M1 Pro

Introducing LLaMA voice chat! 🦙 You can run this locally on an M1 Pro

Georgi Gerganov

1,682,238 次观看 • 3 年前

Full F16 precision 34B Code Llama at >20 t/s on M2 Ultra

Full F16 precision 34B Code Llama at >20 t/s on M2 Ultra

Georgi Gerganov

1,158,754 次观看 • 3 年前

Casually running a 180B parameter LLM on M2 Ultra

Casually running a 180B parameter LLM on M2 Ultra

Georgi Gerganov

683,423 次观看 • 3 年前

Here is another Gibberlink experiment: Two AI agents autonomously encrypt their audio chat (video by Anton Pidkuiko)

Here is another Gibberlink experiment: Two AI agents autonomously encrypt their audio chat (video by Anton Pidkuiko)

Georgi Gerganov

270,537 次观看 • 1 年前

DeepSeek-R1 on Mac Studio 192GB 🪄

DeepSeek-R1 on Mac Studio 192GB 🪄

Georgi Gerganov

259,545 次观看 • 1 年前

LLaMA voice chat + Siri TTS This example is now truly 100% offline since we are now using the built-in Siri text-to-speech available on MacOS through the "say" command

LLaMA voice chat + Siri TTS This example is now truly 100% offline since we are now using the built-in Siri text-to-speech available on MacOS through the "say" command

Georgi Gerganov

412,007 次观看 • 3 年前

Causally running Grok-1 at home

Causally running Grok-1 at home

Georgi Gerganov

276,954 次观看 • 2 年前

sam.cpp 👀 Inference of Meta's Segment Anything Model on the CPU Project by Yavor Ivanov - powered by

sam.cpp 👀 Inference of Meta's Segment Anything Model on the CPU Project by Yavor Ivanov - powered by

Georgi Gerganov

264,664 次观看 • 3 年前

The Llama app for Mac now comes with a simple request builder for llama.cpp's REST API

The Llama app for Mac now comes with a simple request builder for llama.cpp's REST API

Georgi Gerganov

21,786 次观看 • 1 个月前

Native whisper.cpp server with OAI-like API is now available $ make server && ./server This is a very convenient way to run an efficient local transcription service locally on any kind of hardware (CPU, GPU (CUDA or Metal) or ANE) thx felrock

Native whisper.cpp server with OAI-like API is now available $ make server && ./server This is a very convenient way to run an efficient local transcription service locally on any kind of hardware (CPU, GPU (CUDA or Metal) or ANE) thx felrock

Georgi Gerganov

172,173 次观看 • 2 年前

Here is how to deploy and serve any LLM on HF with a single command in less than 3 minutes with llama.cpp $ bash -c "$(curl -s

Here is how to deploy and serve any LLM on HF with a single command in less than 3 minutes with llama.cpp $ bash -c "$(curl -s

Georgi Gerganov

143,145 次观看 • 2 年前

llama.vscode (powered by Qwen Coder)

llama.vscode (powered by Qwen Coder)

Georgi Gerganov

77,668 次观看 • 1 年前

The example below is using prompt-based speculative decoding. Specifically, ngram hashing is utilized to suggest drafts of up to 64 tokens. The hasher keeps track of ngrams in the observed contexts, so mostly effective for coding tasks. Here is another demo:

The example below is using prompt-based speculative decoding. Specifically, ngram hashing is utilized to suggest drafts of up to 64 tokens. The hasher keeps track of ngrams in the observed contexts, so mostly effective for coding tasks. Here is another demo:

Georgi Gerganov

29,890 次观看 • 6 个月前

GGUF My Repo by Hugging Face Create quantum GGUF models fully online - quickly and secure. Thanks to Vaibhav (VB) Srivastav, Pedro Cuenca and team for creating this HF space! In the video below I give it a try to create a quantum 8-bit model of Gemma 2B - it took about 60 seconds. The resulting model becomes automatically available in your HF profile and is ready to be used with llama.cpp

Georgi Gerganov

64,521 次观看 • 2 年前

Updated the llama.vim plugin to support speculative FIM It now generates the next suggestion while you review the current one. If you accept it, the next suggestion is available immediately.

Updated the llama.vim plugin to support speculative FIM It now generates the next suggestion while you review the current one. If you accept it, the next suggestion is available immediately.

Georgi Gerganov

28,337 次观看 • 1 年前