I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Wrote the full transformer architecture, and BPE tokenizer from scratch. The framework features: - Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x... increased throughput - Automatic WebGPU fallback for non-NVIDIA devices - TypeScript API with Rust compute backend - One npm install to get started, prebuilt binaries for every platform Try out the model for yourself: Built with Reese Chong. Check out the repos and blog if you want to learn more. Shoutout to Modal for the compute credits allowing me to train on 2 A100 GPUs without going broke cc sunny madra Gavinshow more

Everything is open source, and we wrote a thorough blog breaking down everything we did so you guys can learn. Repo: Blog:

Test the LLM and view the probability distribution of next tokens. All running in the browser using our own framework.

As per @ClementDelangue's suggestion, I uploaded the model to HuggingFace. Feel free to check it out and run the model yourself:

you should share the model on

Will most definitely do 🔥

Love it. Dm me?

For sure!

Nice! Would love to get your take on cuTile Rust:

Looks interesting, will play around with it a bit and share my thoughts 👀

DM me if you wanna chat about Rust for GPU programming!

Will definitely reach out, even before I got into AI I did a lot of GPU programming work related to game development and rendering 🔥

cost?

I spent ~$100 CAD of my own money, but @modal gave me ~$400 free credits at a hackathon a while back. The total was $500 of cloud compute

waterloo strikes again

have to live up to the name

how did i get into the same school as this beast

Lmaooo 🙈

first principles 🫡

Yessir 🔥

respect. writing your own framework is one thing, getting a full train run to converge is the part that usually gets ugly. what broke first for you, kernels or optimizer math?

We had a lot of bugs with our CUDA kernels that had to be addressed before the training started to go well 😂

This is so impressive. Can I use your work at my lecture for education purpose ?

@yechan_ai Yes of course, we’re honoured 🔥

What are they feeding Waterloo undergrads

my goats @_reesechong and @MankyDankyBanky this is insane

@_reesechong Ty bro 🙏

fire

Appreciate it, TinyTPU was a big inspo

🐐

nah you inspired me fr

this is crazy bro

very cool, great work!

This is fucking great man!!!

@lucashjin good stuff aadi🔥

@lucashjin Thanks Kaibo looking forward to Bear hacks

very nice!

Amazing

@nicholaschen__ cracked

Well done

@hnshah Part time model builder, full time looksmaxxer

good stuff

holy cracked

thanks moulik!

It's been my dream for long time to do something like that , impressive work Happy for you guyz to see you got much recognize by big giants Just great

nice work!!

Thanks so much Evan!

training a transformer from scratch used to be frontier lab work now random devs ship full implementations with custom CUDA kernels... Rust backends... BPE tokenizers built from zero as side projects what took entire teams 2 years ago is now weekend hobby territory the commodification speed is wild

@imdonghakim w aadi

Super cool stuff, man, super cool.

Thanks man!

Reminds me of what LLMs are doing under the hood.

thats actually insane. most people talk about it, you actually built it. the rust + cuda combo is hard mode too

@anirudhbv_ce W

Please update the demo page so it caches the model on user device it is really cool but please cache it on user device

Good point, will try to get this out soon!