Loading video...

Video Failed to Load

Go Home

I wrote GPU kernels for GPT2 124M inference engine from scratch in ~1500 lines of CUDA. Here I explain it simply.

160,092 views • 2 months ago •via X (Twitter)

41 Comments

Cosmos's profile picture
Cosmos2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Dude... Again a great bomber Btw which tool do you use to edit / create vdos

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Thanks brother, there are paid ai video editors

Hassan's profile picture
Hassan2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode any resource you're using to learn esp gpu programming?

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Claude OP

Hassan's profile picture
Hassan2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode thnx mate

Deepanshu Saini's profile picture
Deepanshu Saini2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Again a bomber is dropped

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Thankyou Brother ❤️

Adarsh V. More's profile picture
Adarsh V. More2 months ago

@kirat_tw @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Damnn bro this stuff sounds soo cool and learning a lot for youu Looking forward for more such content

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Will keep posting the whole inference stack as I keep learning, will try to get cracked in this asap

Satoru Gojo's profile picture
Satoru Gojo2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode never stoping engine

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Inference all the way

Ayush's profile picture
Ayush2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Love cuda , worked in isro and built image processing pipe line for synthetic aperture radar

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Woah, you are cracked dude

Ayush's profile picture
Ayush2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Appreciate it man. 1500 lines of hand written CUDA is no joke, most people never go below the library layer. Drop the repo if it's open, want to see your attention kernel

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Yes, video end has the repo link

Himanshu's profile picture
Himanshu2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Bhai this is not tough i mean it when i say , if you how this works then you can literally do it

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Who said It's tough

Himanshu's profile picture
Himanshu2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Read the comments

Rohan Singla (solana summit & devcon)'s profile picture
Rohan Singla (solana summit & devcon)2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @boskykode bhai aap @nvidia mai apply kro

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @boskykode @nvidia 😂😂 bhai thoda halka chlte hai, time hai abhi

Ashutosh's profile picture
Ashutosh2 months ago

@kirat_tw @Adarsh_VMore @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode things are getting real!

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode One project at a time

Bhoomika's profile picture
Bhoomika2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode aap toh cracked ho

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Are nhi nhi

Sneha Pandey's profile picture
Sneha Pandey2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode So cool!!!

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode You are cool, I'm just doing basic coding

Srijan Prasad's profile picture
Srijan Prasad2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode keep it up Brooo !!

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Thanku brother

Aryan Srivastava's profile picture
Aryan Srivastava2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode How much time it takes to create these videos?

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Bro 😂, just don't ask this

Aryan Srivastava's profile picture
Aryan Srivastava2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Just spill it bro 😂😂😂

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Some paid ai editor, that's all I can reveal. Trade secrets.

Aryan Srivastava's profile picture
Aryan Srivastava2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Btw really nice video and explainations 👍👌

Cooper Veit's profile picture
Cooper Veit2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode legend

Sundaram's profile picture
Sundaram2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode sahil bhai day by day 🚀

ninjaHattori1's profile picture
ninjaHattori12 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode One concept at a time

Ezhumalai CSC's profile picture
Ezhumalai CSC2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Can you have youtube channel

Ravi raj's profile picture
Ravi raj2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Is it cudaboost?

Anirudh Malik's profile picture
Anirudh Malik2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Are you working somewhere?

Arin's profile picture
Arin2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode writing the inference engine was already hard. writing the gpu kernels too is where this gets properly impressive

Bonolo Masima's profile picture
Bonolo Masima2 months ago

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode awesome work Sahil 🔥 I’m wondering if bank conflicts showed up as a bottleneck once you started tiling? looking forward to the fused kernels + KV attention update

Related Videos

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, sssss, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Sakana AI

1,161,386 views • 1 year ago