Loading video...

Video Failed to Load

Go Home

DiffusionGemma can now run at 2000+ tokens/sec! ⚡ We made local DiffusionGemma inference 1.8× faster. Run it on 18GB RAM via Unsloth Studio. GitHub: Guide:

UnslothAI's profile picture

Unsloth AI

76,748 subscribers

178,115 views • 29 days ago •via X (Twitter)

EducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

You can now run GPT, Claude & other models in Unsloth. Connect + run APIs in a local UI: - Code execution, web search, image gen, editing - Auto prompt caching to save costs - Provider features like cites, sandboxes GitHub: Guide:
0:36

You can now run GPT, Claude & other models in Unsloth. Connect + run APIs in a local UI: - Code execution, web search, image gen, editing - Auto prompt caching to save costs - Provider features like cites, sandboxes GitHub: Guide:

Unsloth AI

32,004 views • 1 month ago

Inference in Unsloth Studio is now ~20% faster. You can also use older pre-downloaded GGUFs from Hugging Face etc. AMD chat support for Linux now works. Data Recipes now works on macOS, AMD, CPU setups. GitHub: Changelog:
0:25

Inference in Unsloth Studio is now ~20% faster. You can also use older pre-downloaded GGUFs from Hugging Face etc. AMD chat support for Linux now works. Data Recipes now works on macOS, AMD, CPU setups. GitHub: Changelog:

Unsloth AI

25,355 views • 3 months ago

You can now fine-tune TTS models with Unsloth! Train, run and save models like Sesame-CSM and OpenAI's Whisper locally with our free notebooks. Unsloth makes TTS training 1.5x faster with 50% less VRAM. GitHub: Docs & Notebooks:
0:29

You can now fine-tune TTS models with Unsloth! Train, run and save models like Sesame-CSM and OpenAI's Whisper locally with our free notebooks. Unsloth makes TTS training 1.5x faster with 50% less VRAM. GitHub: Docs & Notebooks:

Unsloth AI

126,049 views • 1 year ago

GOOGLE JUST MADE EVERY CHATBOT FEEL SLOW Diffusion Gemma 26b doesn’t predict word by word, it generates 256 tokens in parallel using bi-directional attention, like stable diffusion but for language it’s MoE so only 3.8B params activate during inference, fits on a single RTX 4090 with 18GB VRAM and you can run it right now via llama.cpp, the same engine powering every tool in my local llm playbook the whole ecosystem exists for moments like this ↓
0:31

GOOGLE JUST MADE EVERY CHATBOT FEEL SLOW Diffusion Gemma 26b doesn’t predict word by word, it generates 256 tokens in parallel using bi-directional attention, like stable diffusion but for language it’s MoE so only 3.8B params activate during inference, fits on a single RTX 4090 with 18GB VRAM and you can run it right now via llama.cpp, the same engine powering every tool in my local llm playbook the whole ecosystem exists for moments like this ↓

leopardracer

27,140 views • 1 month ago

4-bit Qwen3.6 MTP GGUF managed to search 70+ sites from a single prompt. Try this locally on 20GB RAM via Unsloth Studio. Unsloth now supports auto MTP + speculative decoding & auto-selects the best MTP settings for your device (Mac, CPU, GPU). GitHub:
0:28

4-bit Qwen3.6 MTP GGUF managed to search 70+ sites from a single prompt. Try this locally on 20GB RAM via Unsloth Studio. Unsloth now supports auto MTP + speculative decoding & auto-selects the best MTP settings for your device (Mac, CPU, GPU). GitHub:

Unsloth AI

71,978 views • 1 month ago

2-bit Gemma 4 12B GGUF, only 4.66 GB on disk, managed to cite 15 sites from a single prompt. Try this locally on >6GB RAM via Unsloth Studio. GitHub:
0:48

2-bit Gemma 4 12B GGUF, only 4.66 GB on disk, managed to cite 15 sites from a single prompt. Try this locally on >6GB RAM via Unsloth Studio. GitHub:

Unsloth AI

143,604 views • 1 month ago

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.
0:45

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.

Unsloth AI

1,697,662 views • 3 months ago

2-bit Qwen3.6-27B GGUF made 26 tool calls, triaged 15 GitHub issues and fixed, tested + reproed our repo’s 3 latest issues. 🔥 Try this locally in Unsloth Studio with just 12GB RAM. Studio also has a new look! GitHub:
0:36

2-bit Qwen3.6-27B GGUF made 26 tool calls, triaged 15 GitHub issues and fixed, tested + reproed our repo’s 3 latest issues. 🔥 Try this locally in Unsloth Studio with just 12GB RAM. Studio also has a new look! GitHub:

Unsloth AI

77,982 views • 2 months ago

what if you could instantly clone any object? Gemma 4 is now on Cerebras Inference, running up to 10x faster than GPUs (1,500 tokens/sec). Multimodal generations you can iterate on in real time :)
1:05

what if you could instantly clone any object? Gemma 4 is now on Cerebras Inference, running up to 10x faster than GPUs (1,500 tokens/sec). Multimodal generations you can iterate on in real time :)

Sarah Chieng

32,686 views • 19 days ago

Can Google Gemma DiffusionGemma help fix broken OCR? In theory, denoising tokens in parallel could work better for OCR correction since context is seen upfront? Pointed it at 19th-century newspaper OCR. It corrected better than the autoregressive baseline — at ~8x the speed.
0:35

Can Google Gemma DiffusionGemma help fix broken OCR? In theory, denoising tokens in parallel could work better for OCR correction since context is seen upfront? Pointed it at 19th-century newspaper OCR. It corrected better than the autoregressive baseline — at ~8x the speed.

Daniel van Strien

36,105 views • 1 month ago

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥 > Multilingual - English, Chinese, Korean & Japanese > Cross platform inference w/ llama.cpp > Zero-shot voice cloning > Trained on 5 Billion audio tokens > Qwen 2.5 0.5B LLM backbone > Trained via HF GPU grants Model weights on the hub, you can even run this on a Raspberry Pi! Go run, inference now! 🐐
0:50

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥 > Multilingual - English, Chinese, Korean & Japanese > Cross platform inference w/ llama.cpp > Zero-shot voice cloning > Trained on 5 Billion audio tokens > Qwen 2.5 0.5B LLM backbone > Trained via HF GPU grants Model weights on the hub, you can even run this on a Raspberry Pi! Go run, inference now! 🐐

Vaibhav (VB) Srivastav

44,654 views • 1 year ago

Auto regressive LLMs are officially on notice. run Gemma 4 26B diffusion gguf with llama.cpp Google just dropped DiffusionGemma-26B, and it completely flips how we generate text. instead of predicting words one by one, it generates 256 tokens in parallel using bi-directional attention. its like stable diffusion, but for language. the model starts with random text "noise" and iteratively refines and self-corrects the entire block in real-time to fix formatting and reasoning errors on the fly. since it’s a Mixture of Experts (MoE) that only activates 3.8B parameters during inference, it fits perfectly on consumer hardware. You can run the Q4_K_M quant with an 18GB VRAM budget on a single RTX 3090 or RTX 4090 with exceptional throughput. Tested on Ubuntu 22 with CUDA 13.1 using the cutting edge experimental llama.cpp branch. Here is how to compile and run it with the live terminal denoising visualizer: # 1. Clone & check out the experimental PR (#24423) - 1) git clone && cd llama.cpp -git fetch origin 2) pull/24423/head:diffusiongemma && --git checkout diffusiongemma # 2. Build with CUDA support 1) cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native 2) cmake --build build -j $(nproc) --config Release --target llama-diffusion-cli # 3. Run with live visual denoising (llama.cpp flags) ./build/bin/llama-diffusion-cli \ -m /path/to/diffusiongemma-26B-A4B-it-Q4_K_M.gguf \ -ngl 99 -cnv -n 2048 --diffusion-visual Watch the video below to see the live --diffusion-visual canvas iteratively de noising the prompt output in real time. guide and unsloth's hugging face GGUF model links are in the comments below! Is auto regressive generation officially legacy tech? Let me know what you think.
0:31

Auto regressive LLMs are officially on notice. run Gemma 4 26B diffusion gguf with llama.cpp Google just dropped DiffusionGemma-26B, and it completely flips how we generate text. instead of predicting words one by one, it generates 256 tokens in parallel using bi-directional attention. its like stable diffusion, but for language. the model starts with random text "noise" and iteratively refines and self-corrects the entire block in real-time to fix formatting and reasoning errors on the fly. since it’s a Mixture of Experts (MoE) that only activates 3.8B parameters during inference, it fits perfectly on consumer hardware. You can run the Q4_K_M quant with an 18GB VRAM budget on a single RTX 3090 or RTX 4090 with exceptional throughput. Tested on Ubuntu 22 with CUDA 13.1 using the cutting edge experimental llama.cpp branch. Here is how to compile and run it with the live terminal denoising visualizer: # 1. Clone & check out the experimental PR (#24423) - 1) git clone && cd llama.cpp -git fetch origin 2) pull/24423/head:diffusiongemma && --git checkout diffusiongemma # 2. Build with CUDA support 1) cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native 2) cmake --build build -j $(nproc) --config Release --target llama-diffusion-cli # 3. Run with live visual denoising (llama.cpp flags) ./build/bin/llama-diffusion-cli \ -m /path/to/diffusiongemma-26B-A4B-it-Q4_K_M.gguf \ -ngl 99 -cnv -n 2048 --diffusion-visual Watch the video below to see the live --diffusion-visual canvas iteratively de noising the prompt output in real time. guide and unsloth's hugging face GGUF model links are in the comments below! Is auto regressive generation officially legacy tech? Let me know what you think.

Alok

52,656 views • 1 month ago

Can a $351 mini PC run a 30B+ AI model locally? 👀 We tested the MINISFORUM UM790 Pro with Qwen 35B — and the results might surprise you. 💻 Powered by: • AMD Ryzen™ 9 7940HS • Radeon™ 780M iGPU • 48GB DDR5 RAM (dual-channel) 🚀 Smooth local inference at 20+ tokens/sec (based on our test setup, results may vary) No cloud. No API fees. Just local AI. 🔒 🔗 Learn more: #MINISFORUM #UM790Pro #LocalAI #MiniPC #AI #LLM #Tech
1:21

Can a $351 mini PC run a 30B+ AI model locally? 👀 We tested the MINISFORUM UM790 Pro with Qwen 35B — and the results might surprise you. 💻 Powered by: • AMD Ryzen™ 9 7940HS • Radeon™ 780M iGPU • 48GB DDR5 RAM (dual-channel) 🚀 Smooth local inference at 20+ tokens/sec (based on our test setup, results may vary) No cloud. No API fees. Just local AI. 🔒 🔗 Learn more: #MINISFORUM #UM790Pro #LocalAI #MiniPC #AI #LLM #Tech

MINISFORUM Official

74,557 views • 2 months ago

Can it run doom was yesterday. Today is "can it run Gemma4". It even runs on a Nintendo Switch 1 @ 1.5 t/sec
2:57

Can it run doom was yesterday. Today is "can it run Gemma4". It even runs on a Nintendo Switch 1 @ 1.5 t/sec

Chubby♨️

20,542 views • 3 months ago

Your AI agent is costing you money for no reason. OpenClaw can now run UNLIMITED for free. No subscriptions. No token limits. No cloud dependency. All you need: • Max Studio • Ollama • GLM 4.7 Flash Install it once. Run it locally. Unlimited tokens. Most people are still paying monthly for what you can now run free.
0:32

Your AI agent is costing you money for no reason. OpenClaw can now run UNLIMITED for free. No subscriptions. No token limits. No cloud dependency. All you need: • Max Studio • Ollama • GLM 4.7 Flash Install it once. Run it locally. Unlimited tokens. Most people are still paying monthly for what you can now run free.

Julian Goldie SEO

44,309 views • 4 months ago

Cerebras Inference powers the world's top models - like Qwen3 Coder at 2000 tokens/s 🤯 And you can now access these models directly in Visual Studio Code with the extension and an API key (that you can get for free from
0:22

Cerebras Inference powers the world's top models - like Qwen3 Coder at 2000 tokens/s 🤯 And you can now access these models directly in Visual Studio Code with the extension and an API key (that you can get for free from

Visual Studio Code

57,360 views • 9 months ago

Ram Mahal Dhaba at Kurla station Run by Rahim? Can someone confirm it? 😳
0:28

Ram Mahal Dhaba at Kurla station Run by Rahim? Can someone confirm it? 😳

Kreately.in

85,099 views • 4 months ago

Can You Run Faster On All Fours? 🤔
0:27

Can You Run Faster On All Fours? 🤔

Zack D. Films

49,965 views • 1 month ago

Llama 4 Scout 109B can actually run pretty fast on a 64GB M4 Max using a mixed 3/6 bit quant (4.0 bits-per-weight). Here it is generating 600+ tokens at 42 tok/sec on my laptop!
0:22

Llama 4 Scout 109B can actually run pretty fast on a 64GB M4 Max using a mixed 3/6 bit quant (4.0 bits-per-weight). Here it is generating 600+ tokens at 42 tok/sec on my laptop!

Awni Hannun

39,146 views • 1 year ago