Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

DiffusionGemma can now run at 2000+ tokens/sec! ⚡ We made local DiffusionGemma inference 1.8× faster. Run it on 18GB RAM via Unsloth Studio. GitHub: Guide:

UnslothAI's profile picture

Unsloth AI

87,262 subscribers

179,979 görüntüleme • 2 ay önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

You can now run GPT, Claude & other models in Unsloth. Connect + run APIs in a local UI: - Code execution, web search, image gen, editing - Auto prompt caching to save costs - Provider features like cites, sandboxes GitHub: Guide:
0:36

You can now run GPT, Claude & other models in Unsloth. Connect + run APIs in a local UI: - Code execution, web search, image gen, editing - Auto prompt caching to save costs - Provider features like cites, sandboxes GitHub: Guide:

Unsloth AI

32,413 görüntüleme • 2 ay önce

Inference in Unsloth Studio is now ~20% faster. You can also use older pre-downloaded GGUFs from Hugging Face etc. AMD chat support for Linux now works. Data Recipes now works on macOS, AMD, CPU setups. GitHub: Changelog:
0:25

Inference in Unsloth Studio is now ~20% faster. You can also use older pre-downloaded GGUFs from Hugging Face etc. AMD chat support for Linux now works. Data Recipes now works on macOS, AMD, CPU setups. GitHub: Changelog:

Unsloth AI

25,355 görüntüleme • 4 ay önce

You can now fine-tune TTS models with Unsloth! Train, run and save models like Sesame-CSM and OpenAI's Whisper locally with our free notebooks. Unsloth makes TTS training 1.5x faster with 50% less VRAM. GitHub: Docs & Notebooks:
0:29

You can now fine-tune TTS models with Unsloth! Train, run and save models like Sesame-CSM and OpenAI's Whisper locally with our free notebooks. Unsloth makes TTS training 1.5x faster with 50% less VRAM. GitHub: Docs & Notebooks:

Unsloth AI

126,494 görüntüleme • 1 yıl önce

GOOGLE JUST MADE EVERY CHATBOT FEEL SLOW Diffusion Gemma 26b doesn’t predict word by word, it generates 256 tokens in parallel using bi-directional attention, like stable diffusion but for language it’s MoE so only 3.8B params activate during inference, fits on a single RTX 4090 with 18GB VRAM and you can run it right now via llama.cpp, the same engine powering every tool in my local llm playbook the whole ecosystem exists for moments like this ↓
0:31

GOOGLE JUST MADE EVERY CHATBOT FEEL SLOW Diffusion Gemma 26b doesn’t predict word by word, it generates 256 tokens in parallel using bi-directional attention, like stable diffusion but for language it’s MoE so only 3.8B params activate during inference, fits on a single RTX 4090 with 18GB VRAM and you can run it right now via llama.cpp, the same engine powering every tool in my local llm playbook the whole ecosystem exists for moments like this ↓

leopardracer

27,279 görüntüleme • 2 ay önce

4-bit Qwen3.6 MTP GGUF managed to search 70+ sites from a single prompt. Try this locally on 20GB RAM via Unsloth Studio. Unsloth now supports auto MTP + speculative decoding & auto-selects the best MTP settings for your device (Mac, CPU, GPU). GitHub:
0:28

4-bit Qwen3.6 MTP GGUF managed to search 70+ sites from a single prompt. Try this locally on 20GB RAM via Unsloth Studio. Unsloth now supports auto MTP + speculative decoding & auto-selects the best MTP settings for your device (Mac, CPU, GPU). GitHub:

Unsloth AI

72,279 görüntüleme • 2 ay önce

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.
0:45

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.

Unsloth AI

1,735,159 görüntüleme • 4 ay önce

NVIDIA'S $30K H100 JUST GOT OUT-INFERENCED BY A $250 RYZEN. 18 TOKENS/SEC ON A 35B MODEL, $0.03 IN ELECTRICITY, $47B GONE FROM NVDA BY FRIDAY download qwen3 35b-a3b -> load in lm studio -> disable gpu entirely -> run on cpu + ram -> hit 18 tokens/sec on a 6-core ryzen -> pay nothing per token. that loop is why moe models just made every consumer cpu a viable ai host and nvidia's inference monopoly is quietly on borrowed time. qwen3 35b-a3b + lm studio + ryzen 5 9600x + 32gb ddr5 + zero gpu - that's the stack. watch and save it, then run a 35b model on your laptop this weekend.
6:33

NVIDIA'S $30K H100 JUST GOT OUT-INFERENCED BY A $250 RYZEN. 18 TOKENS/SEC ON A 35B MODEL, $0.03 IN ELECTRICITY, $47B GONE FROM NVDA BY FRIDAY download qwen3 35b-a3b -> load in lm studio -> disable gpu entirely -> run on cpu + ram -> hit 18 tokens/sec on a 6-core ryzen -> pay nothing per token. that loop is why moe models just made every consumer cpu a viable ai host and nvidia's inference monopoly is quietly on borrowed time. qwen3 35b-a3b + lm studio + ryzen 5 9600x + 32gb ddr5 + zero gpu - that's the stack. watch and save it, then run a 35b model on your laptop this weekend.

Antid

23,745 görüntüleme • 17 gün önce

what if you could instantly clone any object? Gemma 4 is now on Cerebras Inference, running up to 10x faster than GPUs (1,500 tokens/sec). Multimodal generations you can iterate on in real time :)
1:05

what if you could instantly clone any object? Gemma 4 is now on Cerebras Inference, running up to 10x faster than GPUs (1,500 tokens/sec). Multimodal generations you can iterate on in real time :)

Sarah Chieng

32,811 görüntüleme • 1 ay önce

Auto regressive LLMs are officially on notice. run Gemma 4 26B diffusion gguf with llama.cpp Google just dropped DiffusionGemma-26B, and it completely flips how we generate text. instead of predicting words one by one, it generates 256 tokens in parallel using bi-directional attention. its like stable diffusion, but for language. the model starts with random text "noise" and iteratively refines and self-corrects the entire block in real-time to fix formatting and reasoning errors on the fly. since it’s a Mixture of Experts (MoE) that only activates 3.8B parameters during inference, it fits perfectly on consumer hardware. You can run the Q4_K_M quant with an 18GB VRAM budget on a single RTX 3090 or RTX 4090 with exceptional throughput. Tested on Ubuntu 22 with CUDA 13.1 using the cutting edge experimental llama.cpp branch. Here is how to compile and run it with the live terminal denoising visualizer: # 1. Clone & check out the experimental PR (#24423) - 1) git clone && cd llama.cpp -git fetch origin 2) pull/24423/head:diffusiongemma && --git checkout diffusiongemma # 2. Build with CUDA support 1) cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native 2) cmake --build build -j $(nproc) --config Release --target llama-diffusion-cli # 3. Run with live visual denoising (llama.cpp flags) ./build/bin/llama-diffusion-cli \ -m /path/to/diffusiongemma-26B-A4B-it-Q4_K_M.gguf \ -ngl 99 -cnv -n 2048 --diffusion-visual Watch the video below to see the live --diffusion-visual canvas iteratively de noising the prompt output in real time. guide and unsloth's hugging face GGUF model links are in the comments below! Is auto regressive generation officially legacy tech? Let me know what you think.
0:31

Auto regressive LLMs are officially on notice. run Gemma 4 26B diffusion gguf with llama.cpp Google just dropped DiffusionGemma-26B, and it completely flips how we generate text. instead of predicting words one by one, it generates 256 tokens in parallel using bi-directional attention. its like stable diffusion, but for language. the model starts with random text "noise" and iteratively refines and self-corrects the entire block in real-time to fix formatting and reasoning errors on the fly. since it’s a Mixture of Experts (MoE) that only activates 3.8B parameters during inference, it fits perfectly on consumer hardware. You can run the Q4_K_M quant with an 18GB VRAM budget on a single RTX 3090 or RTX 4090 with exceptional throughput. Tested on Ubuntu 22 with CUDA 13.1 using the cutting edge experimental llama.cpp branch. Here is how to compile and run it with the live terminal denoising visualizer: # 1. Clone & check out the experimental PR (#24423) - 1) git clone && cd llama.cpp -git fetch origin 2) pull/24423/head:diffusiongemma && --git checkout diffusiongemma # 2. Build with CUDA support 1) cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native 2) cmake --build build -j $(nproc) --config Release --target llama-diffusion-cli # 3. Run with live visual denoising (llama.cpp flags) ./build/bin/llama-diffusion-cli \ -m /path/to/diffusiongemma-26B-A4B-it-Q4_K_M.gguf \ -ngl 99 -cnv -n 2048 --diffusion-visual Watch the video below to see the live --diffusion-visual canvas iteratively de noising the prompt output in real time. guide and unsloth's hugging face GGUF model links are in the comments below! Is auto regressive generation officially legacy tech? Let me know what you think.

Alok

52,656 görüntüleme • 2 ay önce

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥 > Multilingual - English, Chinese, Korean & Japanese > Cross platform inference w/ llama.cpp > Zero-shot voice cloning > Trained on 5 Billion audio tokens > Qwen 2.5 0.5B LLM backbone > Trained via HF GPU grants Model weights on the hub, you can even run this on a Raspberry Pi! Go run, inference now! 🐐
0:50

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥 > Multilingual - English, Chinese, Korean & Japanese > Cross platform inference w/ llama.cpp > Zero-shot voice cloning > Trained on 5 Billion audio tokens > Qwen 2.5 0.5B LLM backbone > Trained via HF GPU grants Model weights on the hub, you can even run this on a Raspberry Pi! Go run, inference now! 🐐

Vaibhav (VB) Srivastav

44,654 görüntüleme • 1 yıl önce

Can a $351 mini PC run a 30B+ AI model locally? 👀 We tested the MINISFORUM UM790 Pro with Qwen 35B — and the results might surprise you. 💻 Powered by: • AMD Ryzen™ 9 7940HS • Radeon™ 780M iGPU • 48GB DDR5 RAM (dual-channel) 🚀 Smooth local inference at 20+ tokens/sec (based on our test setup, results may vary) No cloud. No API fees. Just local AI. 🔒 🔗 Learn more: #MINISFORUM #UM790Pro #LocalAI #MiniPC #AI #LLM #Tech
1:21

Can a $351 mini PC run a 30B+ AI model locally? 👀 We tested the MINISFORUM UM790 Pro with Qwen 35B — and the results might surprise you. 💻 Powered by: • AMD Ryzen™ 9 7940HS • Radeon™ 780M iGPU • 48GB DDR5 RAM (dual-channel) 🚀 Smooth local inference at 20+ tokens/sec (based on our test setup, results may vary) No cloud. No API fees. Just local AI. 🔒 🔗 Learn more: #MINISFORUM #UM790Pro #LocalAI #MiniPC #AI #LLM #Tech

MINISFORUM Official

74,557 görüntüleme • 3 ay önce

Introducing Unsloth Desktop 🦥 The first desktop app to run and train models locally. • Open-source. Runs on Mac, Windows and Linux • Supports MLX, diffusion image/video, audio, GGUF • Connect Claude Code and Codex to local LLMs • 50% more accurate, self-healing tool calls + sandboxed code exec • Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac • Train models 2× faster with 70% less VRAM • Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF) • Use Unsloth’s OpenAI-compatible API and cloud models • Securely deploy LLMs remotely and access anywhere Unsloth Desktop is now available on and GitHub. GitHub: Blog and Guide:
0:46

Introducing Unsloth Desktop 🦥 The first desktop app to run and train models locally. • Open-source. Runs on Mac, Windows and Linux • Supports MLX, diffusion image/video, audio, GGUF • Connect Claude Code and Codex to local LLMs • 50% more accurate, self-healing tool calls + sandboxed code exec • Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac • Train models 2× faster with 70% less VRAM • Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF) • Use Unsloth’s OpenAI-compatible API and cloud models • Securely deploy LLMs remotely and access anywhere Unsloth Desktop is now available on and GitHub. GitHub: Blog and Guide:

Unsloth AI

617,430 görüntüleme • 1 gün önce

2-bit Muse Glimmer GGUF managed to call 100+ tools on just 14GB RAM. 🔥 Muse Glimmer did a complete repo bug hunt for 5 mins nonstop with: evidence, repro, fix, tests and a PR writeup. Run and train it in Unsloth. GitHub repo:
1:58

2-bit Muse Glimmer GGUF managed to call 100+ tools on just 14GB RAM. 🔥 Muse Glimmer did a complete repo bug hunt for 5 mins nonstop with: evidence, repro, fix, tests and a PR writeup. Run and train it in Unsloth. GitHub repo:

Unsloth AI

117,212 görüntüleme • 2 gün önce

Your AI agent is costing you money for no reason. OpenClaw can now run UNLIMITED for free. No subscriptions. No token limits. No cloud dependency. All you need: • Max Studio • Ollama • GLM 4.7 Flash Install it once. Run it locally. Unlimited tokens. Most people are still paying monthly for what you can now run free.
0:32

Your AI agent is costing you money for no reason. OpenClaw can now run UNLIMITED for free. No subscriptions. No token limits. No cloud dependency. All you need: • Max Studio • Ollama • GLM 4.7 Flash Install it once. Run it locally. Unlimited tokens. Most people are still paying monthly for what you can now run free.

Julian Goldie SEO

44,309 görüntüleme • 5 ay önce

Cerebras Inference powers the world's top models - like Qwen3 Coder at 2000 tokens/s 🤯 And you can now access these models directly in Visual Studio Code with the extension and an API key (that you can get for free from
0:22

Cerebras Inference powers the world's top models - like Qwen3 Coder at 2000 tokens/s 🤯 And you can now access these models directly in Visual Studio Code with the extension and an API key (that you can get for free from

Visual Studio Code

57,360 görüntüleme • 10 ay önce

Llama 4 Scout 109B can actually run pretty fast on a 64GB M4 Max using a mixed 3/6 bit quant (4.0 bits-per-weight). Here it is generating 600+ tokens at 42 tok/sec on my laptop!
0:22

Llama 4 Scout 109B can actually run pretty fast on a 64GB M4 Max using a mixed 3/6 bit quant (4.0 bits-per-weight). Here it is generating 600+ tokens at 42 tok/sec on my laptop!

Awni Hannun

39,146 görüntüleme • 1 yıl önce

We just made AI that reads your mind and automates for you. TrustAI (TrustAI (YC S26)) understands your context across the browser (CRMs, internal tools, Slack, GitHub) and suggests personalized automations you can run in one click. Accept, tweak, or ignore, then keep working while we handle the busywork in the background. The data stays local, and the automations can run up to 47x faster post onboarding than leading competitors. Automations when you need them, built in seconds, one click to run.
1:26

We just made AI that reads your mind and automates for you. TrustAI (TrustAI (YC S26)) understands your context across the browser (CRMs, internal tools, Slack, GitHub) and suggests personalized automations you can run in one click. Accept, tweak, or ignore, then keep working while we handle the busywork in the background. The data stays local, and the automations can run up to 47x faster post onboarding than leading competitors. Automations when you need them, built in seconds, one click to run.

Hannah Chung

16,951 görüntüleme • 1 ay önce

Letta Code can now run **fully locally** with an embedded server - no login or Docker required Memory is stored locally, but can be synced to GitHub with `/memory-repository` Now includes builtin support for local LLMs (ollama LM Studio) through pi-ai from Mario Zechner
0:34

Letta Code can now run **fully locally** with an embedded server - no login or Docker required Memory is stored locally, but can be synced to GitHub with `/memory-repository` Now includes builtin support for local LLMs (ollama LM Studio) through pi-ai from Mario Zechner

Letta

20,614 görüntüleme • 2 ay önce

DeepSeek-R1-0528 is now live on Hyperbolic’s Serverless Inference! We also are the first to serve the latest DeepSeek model on Hugging Face. 🟣 Run it instantly:
0:20

DeepSeek-R1-0528 is now live on Hyperbolic’s Serverless Inference! We also are the first to serve the latest DeepSeek model on Hugging Face. 🟣 Run it instantly:

Hyperbolic

33,228 görüntüleme • 1 yıl önce

A new Air release is out From now on: add your favorite agents like Github Copilot, Opencode, and 17 others, and run local models through the ACP. With this release Air also gets Java language support, powered by IntelliJ IDEA code engine. Let us guide you through it🧵
0:36

A new Air release is out From now on: add your favorite agents like Github Copilot, Opencode, and 17 others, and run local models through the ACP. With this release Air also gets Java language support, powered by IntelliJ IDEA code engine. Let us guide you through it🧵

JetBrains Air

45,322 görüntüleme • 28 gün önce