I built a GPT2 124M inference engine from scratch in ~1000 lines of C++. Here I explain it simply.

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs 1000 lines of C++ is insane lol. Codex has helped me build some wild stuff but I’m not pretending I understand all that.

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Yeah, 1000 lines cause CUDA is not involved yet, will add soon

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Andrej kar paaji

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs 😂

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Really appreciate how clearly you broke this down. Building a GPT-2 inference engine from scratch is impressive work 👏. Bravo

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thenk you brother

@kirat_tw @Adarsh_VMore @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs If i install it locally can it build project for me

@kirat_tw @Adarsh_VMore @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Gpt2 is a chat llm, so you can run it locally with this code

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs gg

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs wp

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs How’s the performance on your machine surely it will be better on a gpu ?

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Yes currently running on RAM. loads 1.5gb weights into ram and currently giving 5 tokens/s

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Crazy 😎

@Manik_Khajuria_ @kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs sher

@Manik_Khajuria_ @kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thankyou Bhai ♥️

@kirat_tw @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs GPT2 in 1000 lines of code in c++ is crazy 🤯

@Manik_Khajuria_ @kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs good stuff

@Manik_Khajuria_ @kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thanks brother

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Awesome. this is really good.

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs how did you edit that video? using claude?

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs cool

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Absolutely based

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs That's really great

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Great stuff cooked out there.

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs 🍳

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs What is an inference engine

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Training gives you model weights, inference engine is the code for the model

@kirat_tw @Adarsh_VMore @jitendar100x @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs crazy crazy

@kirat_tw @Adarsh_VMore @jitendar100x @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thankyou rohan ❤️

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @AnkurJangid02 @ashu_463 @100xDevs look like bro is cooking some real stuff .

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @AnkurJangid02 @ashu_463 @100xDevs Going deep into Inference one project at a time.

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs That's Crazy!

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs ding ding ding

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs yo

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Trying out myself !!! Thanks for this….

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Great brother

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Prompt tokens: 21 Prefill: 4241.828 ms Cached decode: 3911.133 ms Naive total: 117044.355 ms Cached total: 8152.961 ms Speedup: 14.35605481x Tokens/sec: 4.090886196

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Great work and explanation! Keep it going!

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thanks brother

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs choice of writing this from scratch in C++🤌, you cooked

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs c++ is the best, agreed

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Can u share resources to learn inference engineering please 😢

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Claude or gpt is all you need

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs LFG..

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Is everything handwriting or used ai at somepoint

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Learned with ai, did myself

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs gpt that to in c++ crazy brother, crazy

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Crazier things to come soon

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs 🔥🔥

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Both the ~1000 lines and ur video editing is crazy 🤯

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thanks brother

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Damnnn

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thanks brother

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs keep cooking yo!

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs At it brother.

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs thank you so much !

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs good explaination learned something!! ty

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs brother is cooking

@kirat_tw @Adarsh_VMore @jitendar100x @rohanBuilds @ashutosh_sao @Deepjyoti100x @deepanshuliv @AnkurJangid02 @ashu_463 @100xDevs Thanks brother