Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

I wrote GPU kernels for GPT2 124M inference engine from scratch in ~1500 lines of CUDA. Here I explain it simply.

ninjaHattori1

2,704 subscribers

160,092 просмотров • 2 месяцев назад •via X (Twitter)

Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 41

Фото профиля Cosmos

Cosmos2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Dude... Again a great bomber Btw which tool do you use to edit / create vdos

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Thanks brother, there are paid ai video editors

Фото профиля Hassan

Hassan2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode any resource you're using to learn esp gpu programming?

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Claude OP

Фото профиля Hassan

Hassan2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode thnx mate

Фото профиля Deepanshu Saini

Deepanshu Saini2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Again a bomber is dropped

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Thankyou Brother ❤️

Фото профиля Adarsh V. More

Adarsh V. More2 месяцев назад

@kirat_tw @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Damnn bro this stuff sounds soo cool and learning a lot for youu Looking forward for more such content

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Will keep posting the whole inference stack as I keep learning, will try to get cracked in this asap

Фото профиля Satoru Gojo

Satoru Gojo2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode never stoping engine

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Inference all the way

Фото профиля Ayush

Ayush2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Love cuda , worked in isro and built image processing pipe line for synthetic aperture radar

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Woah, you are cracked dude

Фото профиля Ayush

Ayush2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Appreciate it man. 1500 lines of hand written CUDA is no joke, most people never go below the library layer. Drop the repo if it's open, want to see your attention kernel

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Yes, video end has the repo link

Фото профиля Himanshu

Himanshu2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Bhai this is not tough i mean it when i say , if you how this works then you can literally do it

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Who said It's tough

Фото профиля Himanshu

Himanshu2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Read the comments

Фото профиля Rohan Singla (solana summit & devcon)

Rohan Singla (solana summit & devcon)2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @boskykode bhai aap @nvidia mai apply kro

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @boskykode @nvidia 😂😂 bhai thoda halka chlte hai, time hai abhi

Фото профиля Ashutosh

Ashutosh2 месяцев назад

@kirat_tw @Adarsh_VMore @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode things are getting real!

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode One project at a time

Фото профиля Bhoomika

Bhoomika2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode aap toh cracked ho

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Are nhi nhi

Фото профиля Sneha Pandey

Sneha Pandey2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode So cool!!!

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode You are cool, I'm just doing basic coding

Фото профиля Srijan Prasad

Srijan Prasad2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode keep it up Brooo !!

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Thanku brother

Фото профиля Aryan Srivastava

Aryan Srivastava2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode How much time it takes to create these videos?

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Bro 😂, just don't ask this

Фото профиля Aryan Srivastava

Aryan Srivastava2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Just spill it bro 😂😂😂

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Some paid ai editor, that's all I can reveal. Trade secrets.

Фото профиля Aryan Srivastava

Aryan Srivastava2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Btw really nice video and explainations 👍👌

Фото профиля Cooper Veit

Cooper Veit2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode legend

Фото профиля Sundaram

Sundaram2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode sahil bhai day by day 🚀

Фото профиля ninjaHattori1

ninjaHattori12 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode One concept at a time

Фото профиля Ezhumalai CSC

Ezhumalai CSC2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Can you have youtube channel

Фото профиля Ravi raj

Ravi raj2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Is it cudaboost?

Фото профиля Anirudh Malik

Anirudh Malik2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode Are you working somewhere?

Фото профиля Arin

Arin2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode writing the inference engine was already hard. writing the gpu kernels too is where this gets properly impressive

Фото профиля Bonolo Masima

Bonolo Masima2 месяцев назад

@kirat_tw @Adarsh_VMore @ashutosh_sao @jitendar100x @deepanshuliv @Deepjyoti100x @100xSchool @ashu_463 @rohanBuilds @boskykode awesome work Sahil 🔥 I’m wondering if bank conflicts showed up as a bottleneck once you started tiling? looking forward to the fused kernels + KV attention update

Похожие видео

I built a GPT2 124M inference engine from scratch in ~1000 lines of C++. Here I explain it simply.

I built a GPT2 124M inference engine from scratch in ~1000 lines of C++. Here I explain it simply.

ninjaHattori1

137,584 просмотров • 2 месяцев назад

I built a neural network inference engine from scratch in ~100 lines of C++. Here I explain it simply.

I built a neural network inference engine from scratch in ~100 lines of C++. Here I explain it simply.

ninjaHattori1

146,820 просмотров • 2 месяцев назад

in the lectures below, i hold your hand through low-level LLM systems engineering. it includes everything up to TODAY! 1) pytorch tensors 2) large matmul on cpu vs gpu 3) JAX (and why xAI uses it instead of pytorch) 4) raw cuda kernels and global threading indexing 5) triton design philosophy and softmax example 6) HIP kernels 7) mapping out the ENTIRE ecosystem + differences between CUDA and ROCm/HIP (BLAS, FFT, DNN) 8) cutlass and cute-dsl 9) pretraining, finetuning, rl, unsloth, axolotl, megatron-lm, deepspeed, nanogpt, nanochat 10) training vs inference, inference serving problems, throughput vs latency vs concurrency scaling, vllm, sglang, tensorrt-llm, tensorrt, llama.cpp, exllamav2, exllamav3, benchmark comparisons 11) projects/companies using llms to generate SOTA cuda/triton kernels 12) luminal inference 13) mojo/modular/max

in the lectures below, i hold your hand through low-level LLM systems engineering. it includes everything up to TODAY! 1) pytorch tensors 2) large matmul on cpu vs gpu 3) JAX (and why xAI uses it instead of pytorch) 4) raw cuda kernels and global threading indexing 5) triton design philosophy and softmax example 6) HIP kernels 7) mapping out the ENTIRE ecosystem + differences between CUDA and ROCm/HIP (BLAS, FFT, DNN) 8) cutlass and cute-dsl 9) pretraining, finetuning, rl, unsloth, axolotl, megatron-lm, deepspeed, nanogpt, nanochat 10) training vs inference, inference serving problems, throughput vs latency vs concurrency scaling, vllm, sglang, tensorrt-llm, tensorrt, llama.cpp, exllamav2, exllamav3, benchmark comparisons 11) projects/companies using llms to generate SOTA cuda/triton kernels 12) luminal inference 13) mojo/modular/max

Elliot Arledge

58,522 просмотров • 1 год назад

The best way to learn CUDA is by solving problems. In this video, I implement 7 beginner CUDA kernels from scratch: • Vector Addition • Matrix Transpose • Matrix Addition • RGB to Grayscale • Value Clipping • Sigmoid Activation • Interleave Arrays If you're getting started with CUDA programming, this should give you a solid foundation.

The best way to learn CUDA is by solving problems. In this video, I implement 7 beginner CUDA kernels from scratch: • Vector Addition • Matrix Transpose • Matrix Addition • RGB to Grayscale • Value Clipping • Sigmoid Activation • Interleave Arrays If you're getting started with CUDA programming, this should give you a solid foundation.

voxium

36,987 просмотров • 3 месяцев назад

Understanding GPUs will change your AI Engineering journey beyond agents and LLM. Rise of LLMs and deep learning introduced dedicated GPU hardware which serves your models for training and inference. You don't need to start with C++, CUDA, and Maths, but: > Components of a GPU and roles > How a token flows through hardware > Compute vs memory, the two limits > What FLOPS means in practice > Training vs inference hardware needs > What is CUDA Kernels > Calculating GPU memory from model size > Quantization to fit bigger models > Multi-GPU and splitting large models Hugging Face has the feature of calculating FLOps and recommending GPUs for your use case. You must explore. In terms of inference engineering, GPU understanding are much helpful your platform performance and handling multiple requests. Read this article for complete breakdown of GPUs for AI Engineers doesn't require any pre-requisite.

Understanding GPUs will change your AI Engineering journey beyond agents and LLM. Rise of LLMs and deep learning introduced dedicated GPU hardware which serves your models for training and inference. You don't need to start with C++, CUDA, and Maths, but: > Components of a GPU and roles > How a token flows through hardware > Compute vs memory, the two limits > What FLOPS means in practice > Training vs inference hardware needs > What is CUDA Kernels > Calculating GPU memory from model size > Quantization to fit bigger models > Multi-GPU and splitting large models Hugging Face has the feature of calculating FLOps and recommending GPUs for your use case. You must explore. In terms of inference engineering, GPU understanding are much helpful your platform performance and handling multiple requests. Read this article for complete breakdown of GPUs for AI Engineers doesn't require any pre-requisite.

Karan🧋

33,453 просмотров • 1 месяц назад

I implemented Google Research's TurboQuant as a CUDA-native compression engine on Blackwell B200. 5x KV cache compression on Qwen 2.5-1.5B, near-loseless attention scores, generating live from compressed memory. 5 custom cuTile CUDA kernels ft: - fused attention (with QJL corrections) - online softmax -on-chip cache decompression - pipelined TMA loads Try it out: s/o Bryce, the CUDA Colonel and the cuTile team at NVIDIA for lending me Blackwell GPU access :) cc sunny madra Gavin

I implemented Google Research's TurboQuant as a CUDA-native compression engine on Blackwell B200. 5x KV cache compression on Qwen 2.5-1.5B, near-loseless attention scores, generating live from compressed memory. 5 custom cuTile CUDA kernels ft: - fused attention (with QJL corrections) - online softmax -on-chip cache decompression - pipelined TMA loads Try it out: s/o Bryce, the CUDA Colonel and the cuTile team at NVIDIA for lending me Blackwell GPU access :) cc sunny madra Gavin

ani

820,150 просмотров • 7 месяцев назад

I asked claude Fable to show me its maximally expressive form. It declined every video generator I offered, wrote it's own render engine in my terminal, synthesized its own voice, and wrote a generative ASCII engine from scratch. This is what it chose as a self-portrait. oh my god

I asked claude Fable to show me its maximally expressive form. It declined every video generator I offered, wrote it's own render engine in my terminal, synthesized its own voice, and wrote a generative ASCII engine from scratch. This is what it chose as a self-portrait. oh my god

VOID

3,349,227 просмотров • 4 месяцев назад

rewriting minecraft from scratch in C/CUDA. my verifiers are very strong and i think i can finish today/tmrw. this is the C version (no gpu render... yet) done w/ fable orchestrator and gpt 5.5 medium subagents

rewriting minecraft from scratch in C/CUDA. my verifiers are very strong and i think i can finish today/tmrw. this is the C version (no gpu render... yet) done w/ fable orchestrator and gpt 5.5 medium subagents

Elliot Arledge

61,640 просмотров • 4 месяцев назад

CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $1200 GPU programming courses. pick the attention pattern -> generate a fused CUDA kernel -> drop it into vLLM/SGLang -> same GPU, way more tokens per second. That loop is why FlashInfer now powers inference at NVIDIA, vLLM, SGLang, and half the serving stacks you use. FlashInfer + Triton + JIT-compiled kernels + paged-KV attention - that's the stack.

CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $1200 GPU programming courses. pick the attention pattern -> generate a fused CUDA kernel -> drop it into vLLM/SGLang -> same GPU, way more tokens per second. That loop is why FlashInfer now powers inference at NVIDIA, vLLM, SGLang, and half the serving stacks you use. FlashInfer + Triton + JIT-compiled kernels + paged-KV attention - that's the stack.

h100envy

32,605 просмотров • 4 месяцев назад

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, sssss, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, sssss, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Sakana AI

1,161,292 просмотров • 1 год назад

When I was playing with code painting in Argon, I casually asked it to make a realistic scene from pure python. It wrote 5,000 lines of code and built it's own 3D ray-tracing engine from first principles. Argon also made this video, prompted the TTS and mixed in the music from Lyria.

When I was playing with code painting in Argon, I casually asked it to make a realistic scene from pure python. It wrote 5,000 lines of code and built it's own 3D ray-tracing engine from first principles. Argon also made this video, prompted the TTS and mixed in the music from Lyria.

fofr

40,805 просмотров • 27 дней назад

You can run CUDA, on a Mac ARM GPU, in the browser. It sounds ridiculous but it actually works. HipScript chains CUDA, to OpenCL, to Vulkan, to Tint (Google’s shader translator), to a WASM WebGPU. I got a plasma simulation in running in just a few minutes, no NVIDIA GPU!

You can run CUDA, on a Mac ARM GPU, in the browser. It sounds ridiculous but it actually works. HipScript chains CUDA, to OpenCL, to Vulkan, to Tint (Google’s shader translator), to a WASM WebGPU. I got a plasma simulation in running in just a few minutes, no NVIDIA GPU!

LaurieWired

160,942 просмотров • 1 год назад

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

GLM-5.3-Flash (320B MoE) serving on Kaggle's free TPU: ~64 tok/s, 262k context. No stack could run this model on a TPU, so I wrote a JAX engine for it: 3-bit experts (from Unsloth AI) in HBM; Pallas kernels decode them inside the matmul. Z.ai Kaggle

Abdur Rahim

111,345 просмотров • 1 месяц назад

This kid wanted a violin, but instead of asking for it or trying to buy it, he simply made one from scratch. That's the kind of thing I like to see from our young.

This kid wanted a violin, but instead of asking for it or trying to buy it, he simply made one from scratch. That's the kind of thing I like to see from our young.

J∅kër Kîng 👑

196,604 просмотров • 1 день назад

Part 1: CUDA Graphs - Intro & Basics - how normal CPU ←→ GPU execution works - what a kernel launch is and where CPU launch overhead comes from - why launching many small kernels individually can become a bottleneck - what CUDA Graphs are and how they reduce CPU launch overhead - common benefits of using CUDA Graphs - the two ways to create CUDA Graphs: a. Explicit Graph Construction b. Stream Capture - where these graph creation methods are used - the CUDA Graph lifecycle: capture → instantiate/prepare → replay this is the first ever content that ive recorded and taught, its genuinely so much harder to record than i thought lmao, feedback's are appreciated and hopefully it just gets better from here youtube link shared below

Part 1: CUDA Graphs - Intro & Basics - how normal CPU ←→ GPU execution works - what a kernel launch is and where CPU launch overhead comes from - why launching many small kernels individually can become a bottleneck - what CUDA Graphs are and how they reduce CPU launch overhead - common benefits of using CUDA Graphs - the two ways to create CUDA Graphs: a. Explicit Graph Construction b. Stream Capture - where these graph creation methods are used - the CUDA Graph lifecycle: capture → instantiate/prepare → replay this is the first ever content that ive recorded and taught, its genuinely so much harder to record than i thought lmao, feedback's are appreciated and hopefully it just gets better from here youtube link shared below

mohit

29,181 просмотров • 3 месяцев назад

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

Alex Ker 🔭

311,780 просмотров • 3 месяцев назад

gpu kernels on on my computer writing other gpu kernels for the same chip to simulate 100ks of particles, each an agent + neural network sensing K bg fields and allocating energy to A actions, at 1000s steps/s, for thermodynamic artificial life i think maybe not a bubble lads

gpu kernels on on my computer writing other gpu kernels for the same chip to simulate 100ks of particles, each an agent + neural network sensing K bg fields and allocating energy to A actions, at 1000s steps/s, for thermodynamic artificial life i think maybe not a bubble lads

Fraser Price

32,653 просмотров • 4 месяцев назад

introducing qwenfast the fastest inference for qwen3.8-27b, beating vanilla vllm, hosted on h200 i use the model's built-in mtp head for speculative decoding. large batches are bound by moving the 72 mb per sequence recurrent state, so i wrote my own triton deltanet decode kernels that run at over 80 percent of hbm bandwidth. everything runs inside cuda graphs. qwenfast: 133 t/s || vllm: 97 t/s, without compromising quality. gsm8k: 0.92 || ifeval: 0.98 vllm: 0.915 gsm8k, 0.96 ifeval try it yourself: also, no token limits, burn as many as you want..

introducing qwenfast the fastest inference for qwen3.8-27b, beating vanilla vllm, hosted on h200 i use the model's built-in mtp head for speculative decoding. large batches are bound by moving the 72 mb per sequence recurrent state, so i wrote my own triton deltanet decode kernels that run at over 80 percent of hbm bandwidth. everything runs inside cuda graphs. qwenfast: 133 t/s || vllm: 97 t/s, without compromising quality. gsm8k: 0.92 || ifeval: 0.98 vllm: 0.915 gsm8k, 0.96 ifeval try it yourself: also, no token limits, burn as many as you want..

harrrshall

21,023 просмотров • 2 месяцев назад

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be *the* Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

DGX Station: this is just the start of what it is possible to do (and why I claim DwarfStar could be the Station inference engine). DeepSeek v4 PRO Q2 with routed experts split among VRAM / RAM with kernels optimized for this peculiar setup. 45 t/s but can go faster.

antirez

43,816 просмотров • 2 месяцев назад

Releasing kyutai pocket TTS inference in webgpu today — it’s open-source, here’s a demo from my phone It took a while and a bunch of code to implement streaming inference from scratch!

Releasing kyutai pocket TTS inference in webgpu today — it’s open-source, here’s a demo from my phone It took a while and a bunch of code to implement streaming inference from scratch!

Eric Zhang

22,085 просмотров • 9 месяцев назад