Загрузка видео...

Не удалось загрузить видео

На главную

🧵1/2 Andrej Karpathy at GPU MODE workshop on llm.c! 👇Full Video

7etsuo's profile picture

tetsuo.ai 💹🧲

83,939 subscribers

16,177 просмотров • 1 год назад •via X (Twitter)

Новости и политикаНаука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 8

Фото профиля tetsuo.ai 💹🧲
tetsuo.ai 💹🧲1 год назад

LLMs in simple, pure C/CUDA: Neural Networks: Zero to Hero: Full Video:

Фото профиля UserInterface
UserInterface4 лет назад

Need Professional Video Production, Music Videos, Commercials, Graphic Design, or Photo Retouching? We will take your project from concept to completion. #services #creative #DMV

Фото профиля Daniel O'Leary
Daniel O'Leary1 год назад

$TETSUO 💪

Фото профиля Bui Dinh Ngoc
Bui Dinh Ngoc1 год назад

This is so cool!

Фото профиля tetsuo.ai 💹🧲
tetsuo.ai 💹🧲1 год назад

yeah it is!

Фото профиля maxwellsdemon⏳
maxwellsdemon⏳1 год назад

he mentions at the end of the talk i think using LLMs as intermediate compilers which generate application specific llm.c files to accelerate workloads instead of relying on cuda/ptx (or high level apis like triton)

Фото профиля Jared / eacc
Jared / eacc1 год назад

this is awesome

Фото профиля Jared / eacc
Jared / eacc1 год назад

how can i explain these to simple people?.....

Похожие видео

CUDA MODE hackathon today! Here's Andrej Karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.
23:28

CUDA MODE hackathon today! Here's Andrej Karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.

swyx

97,440 просмотров • 1 год назад

Andrej Karpathy on how to fight entropy at startups:
1:48

Andrej Karpathy on how to fight entropy at startups:

Z Fellows

26,368 просмотров • 1 год назад

Andrej Karpathy's (Andrej Karpathy) keynote yesterday at AI Startup School in San Francisco.
39:31

Andrej Karpathy's (Andrej Karpathy) keynote yesterday at AI Startup School in San Francisco.

Y Combinator

2,133,291 просмотров • 1 год назад

flash attention explained by Andrej Andrej Karpathy
5:37

flash attention explained by Andrej Andrej Karpathy

ℏεsam

168,185 просмотров • 1 год назад

Andrej Karpathy on Tensor Cores and TF32 precision
4:31

Andrej Karpathy on Tensor Cores and TF32 precision

ℏεsam

112,979 просмотров • 1 год назад

56,000+ tokens/sec at just 80 MHz. 🤯 I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running Andrej Karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT 👇
0:24

56,000+ tokens/sec at just 80 MHz. 🤯 I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running Andrej Karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT 👇

Fabio Guzman

655,852 просмотров • 3 дней назад

Andrej @Karpathy explains on the Dwarkesh Patel podcast why he's not impressed by demos anymore Full episode:
4:28

Andrej @Karpathy explains on the Dwarkesh Patel podcast why he's not impressed by demos anymore Full episode:

Whole Mars Catalog

37,615 просмотров • 3 месяцев назад

Greg and Andrej Karpathy attemping to use grill at OpenAI 2017 offsite
0:28

Greg and Andrej Karpathy attemping to use grill at OpenAI 2017 offsite

Yaroslav Bulatov

818,220 просмотров • 3 месяцев назад

Flash Attention explained by Andrej Karpathy
5:35

Flash Attention explained by Andrej Karpathy

ℏεsam

105,390 просмотров • 1 год назад

Andrej Karpathy (Andrej Karpathy) — co-founded OpenAI, led AI at Tesla, coined "vibe coding." In 4 minutes he explains why software is changing - and why Claude Skills, MCP servers, and AI agents aren't hype anymore. They're the foundation of how software gets built from now on. Imo, worth every second (i've added subtitles)👇
4:22

Andrej Karpathy (Andrej Karpathy) — co-founded OpenAI, led AI at Tesla, coined "vibe coding." In 4 minutes he explains why software is changing - and why Claude Skills, MCP servers, and AI agents aren't hype anymore. They're the foundation of how software gets built from now on. Imo, worth every second (i've added subtitles)👇

darkzodchi

453,653 просмотров • 2 месяцев назад

Andrej Karpathy admits he’s struggling with AI
8:02

Andrej Karpathy admits he’s struggling with AI

Mo

331,101 просмотров • 29 дней назад

Andrej Karpathy Professor Tolkien on what The Lord of The Rings is all about:
1:03

Andrej Karpathy Professor Tolkien on what The Lord of The Rings is all about:

𝐑𝐮𝐠𝐠𝐚

13,821 просмотров • 10 месяцев назад

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply
23:16

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply

Andrej Karpathy

335,861 просмотров • 1 год назад

Just posted video of my talk on "DeFi in the MEV Era" at the recent @Paradigm research workshop. 🧵
35:46

Just posted video of my talk on "DeFi in the MEV Era" at the recent @Paradigm research workshop. 🧵

ciamac moallemi

19,634 просмотров • 1 год назад

Recent from No Priors with Andrej Karpathy on Tesla and Waymo in self-driving🤔
1:48

Recent from No Priors with Andrej Karpathy on Tesla and Waymo in self-driving🤔

Elad Gil

32,448 просмотров • 1 год назад

Andrej Karpathy (OpenAI co-founder) on AI's weirdest blind spot: jokes haven't improved in 4 years.
0:29

Andrej Karpathy (OpenAI co-founder) on AI's weirdest blind spot: jokes haven't improved in 4 years.

Big Brain AI

39,304 просмотров • 1 месяц назад

we got Andrej Karpathy to drive our self-driving golf cart!
3:13

we got Andrej Karpathy to drive our self-driving golf cart!

Georg von Manstein

132,476 просмотров • 1 месяц назад

Inspired by Andrej Karpathy :) A local markdown editor + terminal agent workspace.
4:38

Inspired by Andrej Karpathy :) A local markdown editor + terminal agent workspace.

killian

95,424 просмотров • 1 месяц назад

Andrej Karpathy, Eureka Labs founder & former Director of AI at Tesla, breaks down how LLMs like ChatGPT "download the Internet." Watch the full beginner-friendly breakdown here:
1:07

Andrej Karpathy, Eureka Labs founder & former Director of AI at Tesla, breaks down how LLMs like ChatGPT "download the Internet." Watch the full beginner-friendly breakdown here:

MIT CSAIL

232,929 просмотров • 6 месяцев назад