Загрузка видео...

Не удалось загрузить видео

На главную

PyTorch core engineer at Meta turned CUDA kernel writing into a sport in 13 minutes - better than $1500 GPU programming bootcamps. profile the kernel -> find the bottleneck -> rewrite -> benchmark -> merge the winning code into PyTorch. That loop is how the open community now beats...

35,390 просмотров • 15 дней назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, Suuun, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Sakana AI

1,149,339 просмотров • 1 год назад

Linux is the most widespread operating system, globally – but how is it built? Few people are better to answer this than Greg Kroah-Hartman (Greg K-H): Greg has been a Linux kernel maintainer for 25 years, and one of the 3 Linux Kernel Foundation Fellows (the other two are Linus Torvalds and Shuah Khan). Greg manages the Linux kernel’s stable releases, and is a maintainer of multiple kernel subsystems. We cover the inner workings of Linux kernel development, exploring everything from how changes get implemented to why its community-driven approach produces such reliable software. Greg shares insights about the kernel's unique trust model and makes a case for why engineers should contribute to open-source projects. Watch or listen: • YouTube: • Spotify: • Apple: Brought to you buy: • WorkOS — The modern identity platform for B2B SaaS. • Vanta — Automate compliance and simplify security with Vanta. Check out all of our current sponsor and their offerings: ---- One of the most surprising things I learned about Linux: although 4,000 developers contribute to the project from 500 companies per year, and has around 800 kernel maintainers, the project is run with email, git... and that's about it! No dedicated project managers, no regular meetings across the group. It's a truly fascinating and unique model (and only applies to the Linux kernel project: not to Linux distributions.) We go into more detail about this topic with Greg in the podcast as well.

Gergely Orosz

50,945 просмотров • 1 год назад

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. More links in reply

Andrej Karpathy

336,280 просмотров • 1 год назад

i been running Qwen3.5-35B-A3B UD-Q4_K_XL through Claude Code since llama.cpp merged the Anthropic endpoint. configured it in minutes. everything was great. projects grew from single scripts to multifile systems with 8 modules and 3,000+ lines. then the chains started breaking. 3 to 5 minutes of pure autonomy and suddenly it stops. tool call fails. reprompt. it recovers. 2 minutes later it stops again. the model is fine. the harness is the bottleneck. saw a comment suggesting OpenCode. installed it. pointed it at the same localhost endpoint running the same model on the same GPU. the game is different. instead of stopping on a bad tool call it just keeps going. on wrong read it adjusts. if file not found it retries. the flow is unbroken. i watched it plan a refactor across 8 files, read every module, and start building without a single pause. in Claude Code that same task would have stopped 4 times. the tradeoff is sometimes it loops. same tool call repeated because the model loses track of what it already read. but here is the thing. i choose loops over pauses. a loop you can interrupt and redirect. a broken chain stops the flow and you have to reprompt to get it moving again. someone is solving this at the core level and i have a feeling it is the open source community. the fact that i can run this level of autonomous coding intelligence on a single consumer GPU with 24gb VRAM at 112 tokens per second. respect to the chinese labs. respect to the open source builders making this possible.

Sudo su

67,032 просмотров • 4 месяцев назад

Dario Amodei just told software engineers exactly how long they have. Six to twelve months. Amodei: “I have engineers within Anthropic who say I don’t write any code anymore. I just let the model write the code, I edit it, I do the things around it.” The people building the most powerful AI in history have already stopped writing code. That is not a forecast. That is the current working condition inside the lab closest to the frontier. Amodei: “We might be six to 12 months away from when the model is doing most, maybe all, of what SWEs do end-to-end.” The tech industry spent a decade making software engineers its highest-paid, most protected class. That era has a last day now. When a model can execute an entire software build end-to-end, the ability to write syntax stops being a skill. It becomes a credential for a job that no longer exists. Amodei: “And then it’s a question of how fast does that loop close.” That is the sentence everyone skipped. The code was never the hard part. The hard part was everything around it. The model just learned everything around it. Writing the code is already nearly gone. Testing is next. Deployment is next. When all three collapse into a single autonomous execution loop, the machine no longer needs a human in the chain at all. The corporation or sovereign state that closes that loop first does not gain a competitive advantage. It gains a category of speed that biological engineers cannot match, track, or reverse. That is not disruption. That is replacement at a systems level. Amodei is not describing a future disruption. He is describing the current state of his own building. The loop is already closing. The only question is whether you are inside it or outside it when it seals.

Dustin

318,457 просмотров • 4 месяцев назад