Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🧵1/2 Andrej Karpathy at GPU MODE workshop on llm.c! 👇Full Video

7etsuo's profile picture

tetsuo.ai 💹🧲

83,939 subscribers

16,177 Aufrufe • vor 1 Jahr •via X (Twitter)

Nachrichten & PolitikWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von tetsuo.ai 💹🧲
tetsuo.ai 💹🧲vor 1 Jahr

LLMs in simple, pure C/CUDA: Neural Networks: Zero to Hero: Full Video:

Profilbild von UserInterface
UserInterfacevor 4 Jahren

Need Professional Video Production, Music Videos, Commercials, Graphic Design, or Photo Retouching? We will take your project from concept to completion. #services #creative #DMV

Profilbild von Daniel O'Leary
Daniel O'Learyvor 1 Jahr

$TETSUO 💪

Profilbild von Bui Dinh Ngoc
Bui Dinh Ngocvor 1 Jahr

This is so cool!

Profilbild von tetsuo.ai 💹🧲
tetsuo.ai 💹🧲vor 1 Jahr

yeah it is!

Profilbild von maxwellsdemon⏳
maxwellsdemon⏳vor 1 Jahr

he mentions at the end of the talk i think using LLMs as intermediate compilers which generate application specific llm.c files to accelerate workloads instead of relying on cuda/ptx (or high level apis like triton)

Profilbild von Jared / eacc
Jared / eaccvor 1 Jahr

this is awesome

Profilbild von Jared / eacc
Jared / eaccvor 1 Jahr

how can i explain these to simple people?.....

Ähnliche Videos

CUDA MODE hackathon today! Here's Andrej Karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.
23:28

CUDA MODE hackathon today! Here's Andrej Karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.

swyx

97,440 Aufrufe • vor 1 Jahr

Andrej Karpathy on how to fight entropy at startups:
1:48

Andrej Karpathy on how to fight entropy at startups:

Z Fellows

26,368 Aufrufe • vor 1 Jahr

Andrej Karpathy's (Andrej Karpathy) keynote yesterday at AI Startup School in San Francisco.
39:31

Andrej Karpathy's (Andrej Karpathy) keynote yesterday at AI Startup School in San Francisco.

Y Combinator

2,133,291 Aufrufe • vor 1 Jahr

Andrej Karpathy on Tensor Cores and TF32 precision
4:31

Andrej Karpathy on Tensor Cores and TF32 precision

ℏεsam

112,979 Aufrufe • vor 1 Jahr

flash attention explained by Andrej Andrej Karpathy
5:37

flash attention explained by Andrej Andrej Karpathy

ℏεsam

168,185 Aufrufe • vor 1 Jahr

56,000+ tokens/sec at just 80 MHz. 🤯 I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running Andrej Karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT 👇
0:24

56,000+ tokens/sec at just 80 MHz. 🤯 I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running Andrej Karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT 👇

Fabio Guzman

706,093 Aufrufe • vor 4 Tagen

Andrej @Karpathy explains on the Dwarkesh Patel podcast why he's not impressed by demos anymore Full episode:
4:28

Andrej @Karpathy explains on the Dwarkesh Patel podcast why he's not impressed by demos anymore Full episode:

Whole Mars Catalog

37,615 Aufrufe • vor 3 Monaten

Greg and Andrej Karpathy attemping to use grill at OpenAI 2017 offsite
0:28

Greg and Andrej Karpathy attemping to use grill at OpenAI 2017 offsite

Yaroslav Bulatov

818,220 Aufrufe • vor 3 Monaten

Flash Attention explained by Andrej Karpathy
5:35

Flash Attention explained by Andrej Karpathy

ℏεsam

105,390 Aufrufe • vor 1 Jahr

Andrej Karpathy (Andrej Karpathy) — co-founded OpenAI, led AI at Tesla, coined "vibe coding." In 4 minutes he explains why software is changing - and why Claude Skills, MCP servers, and AI agents aren't hype anymore. They're the foundation of how software gets built from now on. Imo, worth every second (i've added subtitles)👇
4:22

Andrej Karpathy (Andrej Karpathy) — co-founded OpenAI, led AI at Tesla, coined "vibe coding." In 4 minutes he explains why software is changing - and why Claude Skills, MCP servers, and AI agents aren't hype anymore. They're the foundation of how software gets built from now on. Imo, worth every second (i've added subtitles)👇

darkzodchi

453,653 Aufrufe • vor 2 Monaten

Andrej Karpathy admits he’s struggling with AI
8:02

Andrej Karpathy admits he’s struggling with AI

Mo

331,101 Aufrufe • vor 1 Monat

Andrej Karpathy Professor Tolkien on what The Lord of The Rings is all about:
1:03

Andrej Karpathy Professor Tolkien on what The Lord of The Rings is all about:

𝐑𝐮𝐠𝐠𝐚

13,821 Aufrufe • vor 10 Monaten

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply
23:16

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply

Andrej Karpathy

335,861 Aufrufe • vor 1 Jahr

Just posted video of my talk on "DeFi in the MEV Era" at the recent @Paradigm research workshop. 🧵
35:46

Just posted video of my talk on "DeFi in the MEV Era" at the recent @Paradigm research workshop. 🧵

ciamac moallemi

19,634 Aufrufe • vor 1 Jahr

Recent from No Priors with Andrej Karpathy on Tesla and Waymo in self-driving🤔
1:48

Recent from No Priors with Andrej Karpathy on Tesla and Waymo in self-driving🤔

Elad Gil

32,448 Aufrufe • vor 1 Jahr

Andrej Karpathy (OpenAI co-founder) on AI's weirdest blind spot: jokes haven't improved in 4 years.
0:29

Andrej Karpathy (OpenAI co-founder) on AI's weirdest blind spot: jokes haven't improved in 4 years.

Big Brain AI

39,304 Aufrufe • vor 1 Monat

we got Andrej Karpathy to drive our self-driving golf cart!
3:13

we got Andrej Karpathy to drive our self-driving golf cart!

Georg von Manstein

132,476 Aufrufe • vor 1 Monat

Inspired by Andrej Karpathy :) A local markdown editor + terminal agent workspace.
4:38

Inspired by Andrej Karpathy :) A local markdown editor + terminal agent workspace.

killian

95,424 Aufrufe • vor 1 Monat

Andrej Karpathy, Eureka Labs founder & former Director of AI at Tesla, breaks down how LLMs like ChatGPT "download the Internet." Watch the full beginner-friendly breakdown here:
1:07

Andrej Karpathy, Eureka Labs founder & former Director of AI at Tesla, breaks down how LLMs like ChatGPT "download the Internet." Watch the full beginner-friendly breakdown here:

MIT CSAIL

232,929 Aufrufe • vor 6 Monaten