Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Gemma 2 2B running in a browser, powered by WebLLM & WebGPU! 🔥 100% local & on-device In less than 24 hours, we've already got the model to the edge! ⚡ Try it out on an HF space below:

Vaibhav (VB) Srivastav

33,340 subscribers

53,790 views • 1 year ago •via X (Twitter)

Education Science & Technology Arts

Anya Rossi• Live Now

Private livecam show

10 Comments

Vaibhav (VB) Srivastav1 year ago

Play with it here:

Vaibhav (VB) Srivastav1 year ago

It also runs on iOS:

Vaibhav (VB) Srivastav1 year ago

And.. Android too:

Yuchen Jin1 year ago

greta to see MLC and TVM on HF space!

Vaibhav (VB) Srivastav1 year ago

Yesss! I love the speed at which they ship! Quite bullish on them 🤗

Xenek Stoehr1 year ago

Hello, I have this working on Chrome (Win 11) & Ryzen 7, 3750h. It didn't see my Nvidia GTX-1650 but webGPU did see my AMD Radeon RX Vega 10 Graphics (2Gb Shared DDR4-2600 Vram through PCIe 3.0 x16). prompt 21, completion 101, prefill 14.89 tokens/sec, decode 3.2 t/s

liuyo1 year ago

Wow this is crazy, good job

Glenn Cameron Jr1 year ago

@jason_mayes

Prashant1 year ago

This is 🔥 nice work

J͓T͓1 year ago

Daaannnnggg this is super cool! Great work!

Related Videos

Fuck yeah! Llama 3.2 3B running on your browser! 100% local, powered by WebGPU & MLC 🦙

Fuck yeah! Llama 3.2 3B running on your browser! 100% local, powered by WebGPU & MLC 🦙

Vaibhav (VB) Srivastav

281,977 views • 1 year ago

Qwen-2.5 on WebGPU 🏎️ • 42 tok/sec for Qwen2.5-Coder-1.5B on Mac ⚡ • Powered by MLC WebLLM and WebGPU 🔥 Watch Qwen2.5-Coder-1.5B build a website entirely in the browser!

Qwen-2.5 on WebGPU 🏎️ • 42 tok/sec for Qwen2.5-Coder-1.5B on Mac ⚡ • Powered by MLC WebLLM and WebGPU 🔥 Watch Qwen2.5-Coder-1.5B build a website entirely in the browser!

Caleb

41,579 views • 1 year ago

Introducing SmolVLM 256M (& 500M): The world’s smallest multimodal model. Designed for efficiency and perfect for on-device applications! 🔥 It’s so small, it can even run 100% locally in your browser on WebGPU! 🤏 Powered by Transformers.js! ⚡️ Try it out yourself! 👇

Introducing SmolVLM 256M (& 500M): The world’s smallest multimodal model. Designed for efficiency and perfect for on-device applications! 🔥 It’s so small, it can even run 100% locally in your browser on WebGPU! 🤏 Powered by Transformers.js! ⚡️ Try it out yourself! 👇

Xenova

24,966 views • 1 year ago

Introducing Phi-3 WebGPU, a private and powerful AI chatbot that runs locally in your browser, powered by 🤗 Transformers.js and onnxruntime-web! 🔒 On-device inference: no data sent to a server ⚡️ WebGPU-accelerated (> 20 t/s) 📥 Model downloaded once and cached Try it out! 👇

Introducing Phi-3 WebGPU, a private and powerful AI chatbot that runs locally in your browser, powered by 🤗 Transformers.js and onnxruntime-web! 🔒 On-device inference: no data sent to a server ⚡️ WebGPU-accelerated (> 20 t/s) 📥 Model downloaded once and cached Try it out! 👇

Xenova

117,665 views • 2 years ago

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js.🤗 Try it out yourself! Demo + source code below 👇

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js.🤗 Try it out yourself! Demo + source code below 👇

Xenova

19,666 views • 1 year ago

NEW: Kokoro v1.0 - 82M parameters, Apache 2.0 Licensed, multilingual TTS model - powered by WebGPU in browser! 🔥 Fully offline, 100% local with support for English, Spanish, French, Italian, Japanese and Mandarin! ⚡ Works directly in your browser with blazingly fast inference backed by ONNXRuntimeWeb npm i kokoro-js is all you need! What are you waiting for? try it out today! 🤗

NEW: Kokoro v1.0 - 82M parameters, Apache 2.0 Licensed, multilingual TTS model - powered by WebGPU in browser! 🔥 Fully offline, 100% local with support for English, Spanish, French, Italian, Japanese and Mandarin! ⚡ Works directly in your browser with blazingly fast inference backed by ONNXRuntimeWeb npm i kokoro-js is all you need! What are you waiting for? try it out today! 🤗

Vaibhav (VB) Srivastav

88,293 views • 1 year ago

NEW: Alibaba just released Qwen 3.5 Small — a family of powerful multimodal models available in a range of sizes (0.8B, 2B, 4B, and 9B parameters). Perfect for on-device applications! They can even run 100% locally in your browser on WebGPU, powered by Transformers.js! 🤯

NEW: Alibaba just released Qwen 3.5 Small — a family of powerful multimodal models available in a range of sizes (0.8B, 2B, 4B, and 9B parameters). Perfect for on-device applications! They can even run 100% locally in your browser on WebGPU, powered by Transformers.js! 🤯

Xenova

102,592 views • 3 months ago

Private AI browser with the OpenClaw agent on free local models Run your agent on Qwen, Gemma, or Nemotron directly in the browser Open source. Private. Runs on your local device

Private AI browser with the OpenClaw agent on free local models Run your agent on Qwen, Gemma, or Nemotron directly in the browser Open source. Private. Runs on your local device

Sigma Browser

113,142 views • 1 month ago

GGUF My Repo by Hugging Face Create quantum GGUF models fully online - quickly and secure. Thanks to Vaibhav (VB) Srivastav, Pedro Cuenca and team for creating this HF space! In the video below I give it a try to create a quantum 8-bit model of Gemma 2B - it took about 60 seconds. The resulting model becomes automatically available in your HF profile and is ready to be used with llama.cpp

Georgi Gerganov

64,486 views • 2 years ago

Introducing Supertonic WebGPU: blazingly fast text-to-speech running 100% locally in your browser. ⚡️ Generate free audiobooks in seconds, no installation required. In this example, I created a 5-hour audiobook of The Great Gatsby in less than 3 minutes! 🤯 Try it yourself! 👇

Introducing Supertonic WebGPU: blazingly fast text-to-speech running 100% locally in your browser. ⚡️ Generate free audiobooks in seconds, no installation required. In this example, I created a 5-hour audiobook of The Great Gatsby in less than 3 minutes! 🤯 Try it yourself! 👇

Xenova

23,465 views • 6 months ago

Introducing Kokoro.js, a new JavaScript library for running Kokoro TTS, an 82 million parameter text-to-speech model, 100% locally in the browser w/ WASM. Powered by 🤗 Transformers.js. WebGPU support coming soon! 👉 npm i kokoro-js 👈 Link to demo (+ sample code) in 🧵

Introducing Kokoro.js, a new JavaScript library for running Kokoro TTS, an 82 million parameter text-to-speech model, 100% locally in the browser w/ WASM. Powered by 🤗 Transformers.js. WebGPU support coming soon! 👉 npm i kokoro-js 👈 Link to demo (+ sample code) in 🧵

Xenova

40,340 views • 1 year ago

Introducing Moonshine Web: real-time speech recognition running 100% locally in your browser! 🚀 Faster and more accurate than Whisper 🔒 Privacy-focused (no data leaves your device) ⚡️ WebGPU accelerated (w/ WASM fallback) 🔥 Powered by ONNX Runtime Web and Transformers.js

Introducing Moonshine Web: real-time speech recognition running 100% locally in your browser! 🚀 Faster and more accurate than Whisper 🔒 Privacy-focused (no data leaves your device) ⚡️ WebGPU accelerated (w/ WASM fallback) 🔥 Powered by ONNX Runtime Web and Transformers.js

Xenova

27,878 views • 1 year ago

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡

Whisper Large v3 Turbo running 100% local on Mac - powered by Apple Neural Engine! 🔥 Brought to you by argmax ⚡

Vaibhav (VB) Srivastav

51,389 views • 1 year ago

made an interactive video model trained on tap-conditioned data.. tap the screen and it generates the next frames in realtime (small enough to run in the browser with webgpu, even on iphone)

made an interactive video model trained on tap-conditioned data.. tap the screen and it generates the next frames in realtime (small enough to run in the browser with webgpu, even on iphone)

sahir

23,151 views • 2 months ago

real-time vision in your browser 🔥 try out YOLO26 for pose estimation and detection built on WebGPU ⚡️

real-time vision in your browser 🔥 try out YOLO26 for pose estimation and detection built on WebGPU ⚡️

merve

46,395 views • 4 months ago

WebGPU is INSANE! 🤯 Here's a 24B parameter model running locally in a web browser, at a blazing ~50 tokens/second on my M4 Max. ⚡️ It's the largest model we've ever run with Transformers.js... and we're not stopping here. Big announcement soon.

WebGPU is INSANE! 🤯 Here's a 24B parameter model running locally in a web browser, at a blazing ~50 tokens/second on my M4 Max. ⚡️ It's the largest model we've ever run with Transformers.js... and we're not stopping here. Big announcement soon.

Xenova

36,159 views • 2 months ago

Cohere transcribe Sota open source transcription model running in the browser :) Weights on Hugging Face link below

Cohere transcribe Sota open source transcription model running in the browser :) Weights on Hugging Face link below

Nick Frosst

190,722 views • 2 months ago

Unlock local, agentic workflows with Gemma 4 12B and Google AI Edge, directly on your laptop. Experience 100% on-device AI: • Generate code in AI Edge Gallery (new to Mac) • Dictate and edit text via AI Edge Eloquent (new to Mac) • Serve Gemma 4 12B locally with LiteRT-LM Dive in:

Unlock local, agentic workflows with Gemma 4 12B and Google AI Edge, directly on your laptop. Experience 100% on-device AI: • Generate code in AI Edge Gallery (new to Mac) • Dictate and edit text via AI Edge Eloquent (new to Mac) • Serve Gemma 4 12B locally with LiteRT-LM Dive in:

Google for Developers

138,230 views • 11 days ago

Apple released FastVLM so I tried vibe coding a video captioning AI app with it took 5 prompts to get a working app in anycoder and deployed it on Hugging Face 85x faster and 3.4x smaller than comparable sized VLMs the deployed app works 100% locally in your browser powered by transformers.js and WebGPU

Apple released FastVLM so I tried vibe coding a video captioning AI app with it took 5 prompts to get a working app in anycoder and deployed it on Hugging Face 85x faster and 3.4x smaller than comparable sized VLMs the deployed app works 100% locally in your browser powered by transformers.js and WebGPU

AK

42,677 views • 9 months ago

✨ Introducing Gemma 3n, available in early preview today. The model uses a cutting-edge architecture optimized for mobile on-device usage. It brings multimodality, super fast inference, and more.

✨ Introducing Gemma 3n, available in early preview today. The model uses a cutting-edge architecture optimized for mobile on-device usage. It brings multimodality, super fast inference, and more.

Google AI Developers

125,226 views • 1 year ago