Fuck yeah! Llama 3.2 3B running on your browser! 100% local, powered by WebGPU & MLC 🦙

Try it out here:

sweet - i think even better on someone else’s browser :)

Nice, but it seems to take forever to load, as I selected Llama-3.2-3B-instruct-q4f321-MLC. Only 20% 272 seconds. I have 500/500 Mbits/s fiber and normally it should take only 40 seconds to load the entire q4 model!

Not sure! But could be a browser/ transient error in loading weights.

it's so good!!! l love being able to download such powerful things and run them without wifi and then make software that calls to these local llms... it's the new arcade game

What about Safari? Sorta sucks to have to use Chrome.

Runs on my 4 year old Laptop with 16 GB of RAM in Chrome. Cool!

@_philschmid Light em up 🔥🔥🔥

@BrianRoemmele

It has finally come