正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

🧵1/2 Andrej Karpathy at GPU MODE workshop on llm.c! 👇Full Video

tetsuo.ai 💹🧲

83,939 subscribers

16,177 次观看 • 1 年前 •via X (Twitter)

新闻政治科学技术教育

Anya Rossi• Live Now

Private livecam show

8 条评论

tetsuo.ai 💹🧲 的头像

tetsuo.ai 💹🧲1 年前

LLMs in simple, pure C/CUDA: Neural Networks: Zero to Hero: Full Video:

UserInterface 的头像

UserInterface4 年前

Need Professional Video Production, Music Videos, Commercials, Graphic Design, or Photo Retouching? We will take your project from concept to completion. #services #creative #DMV

Daniel O'Leary 的头像

Daniel O'Leary1 年前

$TETSUO 💪

Bui Dinh Ngoc 的头像

Bui Dinh Ngoc1 年前

This is so cool!

tetsuo.ai 💹🧲 的头像

tetsuo.ai 💹🧲1 年前

yeah it is!

maxwellsdemon⏳ 的头像

maxwellsdemon⏳1 年前

he mentions at the end of the talk i think using LLMs as intermediate compilers which generate application specific llm.c files to accelerate workloads instead of relying on cuda/ptx (or high level apis like triton)

Jared / eacc 的头像

Jared / eacc1 年前

this is awesome

Jared / eacc 的头像

Jared / eacc1 年前

how can i explain these to simple people?.....

相关视频

CUDA MODE hackathon today! Here's Andrej Karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.

CUDA MODE hackathon today! Here's Andrej Karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.

swyx

97,440 次观看 • 1 年前

Andrej Karpathy on how to fight entropy at startups:

Andrej Karpathy on how to fight entropy at startups:

Z Fellows

26,368 次观看 • 1 年前

Andrej Karpathy's (Andrej Karpathy) keynote yesterday at AI Startup School in San Francisco.

Andrej Karpathy's (Andrej Karpathy) keynote yesterday at AI Startup School in San Francisco.

Y Combinator

2,133,221 次观看 • 1 年前

Andrej Karpathy on Tensor Cores and TF32 precision

Andrej Karpathy on Tensor Cores and TF32 precision

ℏεsam

112,979 次观看 • 1 年前

flash attention explained by Andrej Andrej Karpathy

flash attention explained by Andrej Andrej Karpathy

ℏεsam

168,185 次观看 • 1 年前

56,000+ tokens/sec at just 80 MHz. 🤯 I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running Andrej Karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT 👇

56,000+ tokens/sec at just 80 MHz. 🤯 I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running Andrej Karpathy microGPT, spelling out names on a tiny LCD. This is GateGPT 👇

Fabio Guzman

549,384 次观看 • 2 天前

Andrej @Karpathy explains on the Dwarkesh Patel podcast why he's not impressed by demos anymore Full episode:

Andrej @Karpathy explains on the Dwarkesh Patel podcast why he's not impressed by demos anymore Full episode:

Whole Mars Catalog

37,615 次观看 • 3 个月前

Greg and Andrej Karpathy attemping to use grill at OpenAI 2017 offsite

Greg and Andrej Karpathy attemping to use grill at OpenAI 2017 offsite

Yaroslav Bulatov

817,989 次观看 • 3 个月前

Flash Attention explained by Andrej Karpathy

Flash Attention explained by Andrej Karpathy

ℏεsam

105,390 次观看 • 1 年前

Andrej Karpathy (Andrej Karpathy) — co-founded OpenAI, led AI at Tesla, coined "vibe coding." In 4 minutes he explains why software is changing - and why Claude Skills, MCP servers, and AI agents aren't hype anymore. They're the foundation of how software gets built from now on. Imo, worth every second (i've added subtitles)👇

Andrej Karpathy (Andrej Karpathy) — co-founded OpenAI, led AI at Tesla, coined "vibe coding." In 4 minutes he explains why software is changing - and why Claude Skills, MCP servers, and AI agents aren't hype anymore. They're the foundation of how software gets built from now on. Imo, worth every second (i've added subtitles)👇

darkzodchi

453,653 次观看 • 2 个月前

Andrej Karpathy admits he’s struggling with AI

Andrej Karpathy admits he’s struggling with AI

Mo

331,101 次观看 • 29 天前

Andrej Karpathy Professor Tolkien on what The Lord of The Rings is all about:

Andrej Karpathy Professor Tolkien on what The Lord of The Rings is all about:

𝐑𝐮𝐠𝐠𝐚

13,821 次观看 • 10 个月前

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply

Andrej Karpathy

335,861 次观看 • 1 年前

Just posted video of my talk on "DeFi in the MEV Era" at the recent @Paradigm research workshop. 🧵

Just posted video of my talk on "DeFi in the MEV Era" at the recent @Paradigm research workshop. 🧵

ciamac moallemi

19,634 次观看 • 1 年前

Recent from No Priors with Andrej Karpathy on Tesla and Waymo in self-driving🤔

Recent from No Priors with Andrej Karpathy on Tesla and Waymo in self-driving🤔

Elad Gil

32,448 次观看 • 1 年前

Andrej Karpathy (OpenAI co-founder) on AI's weirdest blind spot: jokes haven't improved in 4 years.

Andrej Karpathy (OpenAI co-founder) on AI's weirdest blind spot: jokes haven't improved in 4 years.

Big Brain AI

39,304 次观看 • 1 个月前

we got Andrej Karpathy to drive our self-driving golf cart!

we got Andrej Karpathy to drive our self-driving golf cart!

Georg von Manstein

132,476 次观看 • 1 个月前

Inspired by Andrej Karpathy :) A local markdown editor + terminal agent workspace.

Inspired by Andrej Karpathy :) A local markdown editor + terminal agent workspace.

killian

95,424 次观看 • 1 个月前

Andrej featuring Hyperbolic in his latest video was not on my bingo card! 🎉 "My favorite place to interact with the base models is the company called Hyperbolic." Thank you Andrej Karpathy for using and mentioning us! Other exciting news: - We surpassed 100k users - Since our GPU market launch in late December, nearly all H100s have been rented out, driving strong ARR growth.

Andrej featuring Hyperbolic in his latest video was not on my bingo card! 🎉 "My favorite place to interact with the base models is the company called Hyperbolic." Thank you Andrej Karpathy for using and mentioning us! Other exciting news: - We surpassed 100k users - Since our GPU market launch in late December, nearly all H100s have been rented out, driving strong ARR growth.

Yuchen Jin

104,768 次观看 • 1 年前