Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels...

SakanaAILabs's profile picture

Sakana AI

138,686 subscribers

1,160,518 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von vittorio
vittoriovor 1 Jahr

japan bros are back again

Profilbild von Globant
Globantvor 1 Jahr

🚀 Agentic AI Systems are changing what AI can do by having the power to act independently. Unlike traditional AI, which needs constant human supervision, these systems can operate more autonomously. Learn how this shift is leading to smarter solutions that can transform industries.➡️ #TechTrends2025

Profilbild von ludwig
ludwigvor 1 Jahr

I’m going to sleep if I wake up to this having 1M+ views I will read the paper tomorrow morning else pls give me a vibe check chat

Profilbild von Bing Xu
Bing Xuvor 1 Jahr

I quickly take a look of their report on phone, there are a few misleading parts: 1. Torch C++ code is not CUDA kernel, it is calling CUDNN under hood. 2. The highlighted example Conv3D GroupNorm, conv code is not generated at all. The speedup doesn’t make sense if numerical is wrong. 3. It claims wmma can be faster than PyTorch (CUBLAS), is definitely wrong. Probably benchmark error.

Profilbild von main
mainvor 1 Jahr

isn't there clearly something wrong with level_1->15_Matmul_for_lower_triangular_matrices? claimed 152.9x speedup for the kernel on the left over the code on the right. really?

Profilbild von Viraat
Viraatvor 1 Jahr

Hey - wondering if you all are only working with large enterprises right now. If not, we’d love to chat! This would be extremely useful to us - we’re building low-bit models to run efficiently on Jetsons. Generating optimized CUDA code for these would be a game-changer!

Profilbild von aizk ✡️
aizk ✡️vor 1 Jahr

A slow 14 seconds in AI developments

Profilbild von Dan Mac
Dan Macvor 1 Jahr

guys seriously I can't take it anymore need to slow down

Profilbild von Kristof
Kristofvor 1 Jahr

Japan is back

Profilbild von Dr Futuro - e/acc
Dr Futuro - e/accvor 1 Jahr

Wow, Japan is back! 🇯🇵

Ähnliche Videos

🎉 Stoked to share The AI CUDA Engineer 👷 - our end-to-end approach for automating the design and optimization of CUDA Kernels using agentic systems. Blog 📰: Paper 📜: WebUI 📈: Dataset 💽: Awesome team work done with Aaditya Prasad 🇺🇸, Suuun, Maxence Faldor, Yujin Tang, hardmaru 🤗
3:40

🎉 Stoked to share The AI CUDA Engineer 👷 - our end-to-end approach for automating the design and optimization of CUDA Kernels using agentic systems. Blog 📰: Paper 📜: WebUI 📈: Dataset 💽: Awesome team work done with Aaditya Prasad 🇺🇸, Suuun, Maxence Faldor, Yujin Tang, hardmaru 🤗

Robert Lange

42,174 Aufrufe • vor 1 Jahr

AI that responds in under 100ms doesn't happen by accident. Together AI VP of Kernels Dan Fu shares how they use NVIDIA CUDA, TensorRT-LLM, Dynamo and Together ATLAS on NVIDIA Blackwell to power ultra-low-latency inference and long context code generation for Cursor. From building megakernels that fuse an entire model's forward pass into a single launch to enabling real-time voice agents, Together AI is optimizing latency at every layer of the stack.
1:51

AI that responds in under 100ms doesn't happen by accident. Together AI VP of Kernels Dan Fu shares how they use NVIDIA CUDA, TensorRT-LLM, Dynamo and Together ATLAS on NVIDIA Blackwell to power ultra-low-latency inference and long context code generation for Cursor. From building megakernels that fuse an entire model's forward pass into a single launch to enabling real-time voice agents, Together AI is optimizing latency at every layer of the stack.

NVIDIA AI Infrastructure

20,633 Aufrufe • vor 1 Monat

Day 14/90 of Inference Engineering I didn't finish the MNIST in CUDA project today, which made me hesitant to post since I didn't want to put out some slop. But in the spirit of being consistent and showing up, here's a concept I've revisited today now that I've started writing CUDA again. This is an animation on the CUDA memory life cycle. Generally speaking, when writing CUDA, memory management goes through the following stages: 1. Allocate memory on the host 2.Initialize / populate the memory 3.Allocate device memory 4.Copy host data to the device 5. Launch the kernel 6. Copy device data back to the host 7. Free the memory - On a brief reflection, I'm starting to write the matmul kernels in CUDA instead of in C. So this in itself is another paradigm shift in thinking. Excited to hone my technique over the coming week. I'll update you guys when I've completed the project; otherwise I'll just make a post like usual on what I've learned during the day!
0:22

Day 14/90 of Inference Engineering I didn't finish the MNIST in CUDA project today, which made me hesitant to post since I didn't want to put out some slop. But in the spirit of being consistent and showing up, here's a concept I've revisited today now that I've started writing CUDA again. This is an animation on the CUDA memory life cycle. Generally speaking, when writing CUDA, memory management goes through the following stages: 1. Allocate memory on the host 2.Initialize / populate the memory 3.Allocate device memory 4.Copy host data to the device 5. Launch the kernel 6. Copy device data back to the host 7. Free the memory - On a brief reflection, I'm starting to write the matmul kernels in CUDA instead of in C. So this in itself is another paradigm shift in thinking. Excited to hone my technique over the coming week. I'll update you guys when I've completed the project; otherwise I'll just make a post like usual on what I've learned during the day!

max fu

42,808 Aufrufe • vor 1 Monat

Cerebras just had the biggest IPO of the year. Founder Andrew Feldman says the 3 most important things he had to convince investors of while doing the roadshow were that demand for inference is going to 1,000,000x, the GPU isn't the only way to do compute, and that the CUDA moat is overstated. What he said: "Jensen said some time ago on Brad Gerstner's podcast that the demand for inference will grow by a 1,000,000x, and nobody believed him. And at the same time, you saw Sam Altman displaying real vision and going out and trying to lock up huge amounts of compute, memory, data centers, and power, because he saw it too." "[We tried] to share what that means — what exponential demand means. And that we're still so early, and yet the demand for AI compute is overwhelming." "The other thing is that there are lots of ways to do this. The GPU isn't the only way. You've got TPUs, Trainium, and us. There are lots of different ways to build a solution here." "And finally — the notion that CUDA is this grand lock-in is overplayed. Gemini 3, which is an excellent model, was trained on TPUs with no CUDA. The Anthropic models were trained on Trainium with no CUDA. Some of the best models, some of the most interesting things are being done without CUDA. And that lock-in might be overplayed." $CBRS
1:46

Cerebras just had the biggest IPO of the year. Founder Andrew Feldman says the 3 most important things he had to convince investors of while doing the roadshow were that demand for inference is going to 1,000,000x, the GPU isn't the only way to do compute, and that the CUDA moat is overstated. What he said: "Jensen said some time ago on Brad Gerstner's podcast that the demand for inference will grow by a 1,000,000x, and nobody believed him. And at the same time, you saw Sam Altman displaying real vision and going out and trying to lock up huge amounts of compute, memory, data centers, and power, because he saw it too." "[We tried] to share what that means — what exponential demand means. And that we're still so early, and yet the demand for AI compute is overwhelming." "The other thing is that there are lots of ways to do this. The GPU isn't the only way. You've got TPUs, Trainium, and us. There are lots of different ways to build a solution here." "And finally — the notion that CUDA is this grand lock-in is overplayed. Gemini 3, which is an excellent model, was trained on TPUs with no CUDA. The Anthropic models were trained on Trainium with no CUDA. Some of the best models, some of the most interesting things are being done without CUDA. And that lock-in might be overplayed." $CBRS

TBPN

36,474 Aufrufe • vor 3 Monaten

We're excited to back Sail Research, the inference platform built for long-horizon, asynchronous agents. The next wave of AI isn't a person typing into a box. It's agents running for minutes or hours, burning orders of magnitude more tokens per task. At that scale, cost and capacity beat raw latency, and teams are already hitting budget caps. Neil Movva and Samir Menon are solving this full-stack, from CUDA kernels to the API. Hear Neil and Aditya Naganath on why inference is being rebuilt for the age of agents.
3:35

We're excited to back Sail Research, the inference platform built for long-horizon, asynchronous agents. The next wave of AI isn't a person typing into a box. It's agents running for minutes or hours, burning orders of magnitude more tokens per task. At that scale, cost and capacity beat raw latency, and teams are already hitting budget caps. Neil Movva and Samir Menon are solving this full-stack, from CUDA kernels to the API. Hear Neil and Aditya Naganath on why inference is being rebuilt for the age of agents.

Kleiner Perkins

17,431 Aufrufe • vor 2 Monaten

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.
2:24

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.

Grummz

30,573 Aufrufe • vor 2 Monaten

PyTorch 2.13 brings FlexAttention to Apple Silicon, cuts peak memory by up to 4× for large-vocabulary models with nn.LinearCrossEntropyLoss, and updates distributed training, compilation, profiling, and on-device inference. Our live Q&A examined CUDA version support, CuTeDSL in TorchInductor, Python 3.15, ExecuTorch, torch.compile, and plans for PyTorch 2.14. Andrey Talman (Meta), albanD (Meta), and Piotr Bialecki (NVIDIA) joined moderator Chris Gottbrath (Gottbrath Technologies). This clip explains how FlexAttention compiles kernels specialized for the requested masking pattern, allowing them to skip unnecessary computation. It also covers speedups of up to approximately 12× over SDPA in tested sparse configurations and a deterministic backward path on CUDA that removes one source of nondeterminism in gradient computation. 🔗 Watch the full PyTorch 2.13 Release Live Q&A:
1:30

PyTorch 2.13 brings FlexAttention to Apple Silicon, cuts peak memory by up to 4× for large-vocabulary models with nn.LinearCrossEntropyLoss, and updates distributed training, compilation, profiling, and on-device inference. Our live Q&A examined CUDA version support, CuTeDSL in TorchInductor, Python 3.15, ExecuTorch, torch.compile, and plans for PyTorch 2.14. Andrey Talman (Meta), albanD (Meta), and Piotr Bialecki (NVIDIA) joined moderator Chris Gottbrath (Gottbrath Technologies). This clip explains how FlexAttention compiles kernels specialized for the requested masking pattern, allowing them to skip unnecessary computation. It also covers speedups of up to approximately 12× over SDPA in tested sparse configurations and a deterministic backward path on CUDA that removes one source of nondeterminism in gradient computation. 🔗 Watch the full PyTorch 2.13 Release Live Q&A:

PyTorch

18,591 Aufrufe • vor 27 Tagen

The CUDA moat is real, but probably not for long, says CEO of AI infrastructure platform Modal Erik Bernhardsson. He says he's bullish on alternative accelerators over the 2-3 year timeline, even though there's currently zero demand from his customers for TPUs, etc. "The cost today of rewriting your software to run on those stacks is very high... But the cost is going to go down." "You're going to have software that basically lets you take CUDA-compatible stuff and run it on alternative accelerators."
1:33

The CUDA moat is real, but probably not for long, says CEO of AI infrastructure platform Modal Erik Bernhardsson. He says he's bullish on alternative accelerators over the 2-3 year timeline, even though there's currently zero demand from his customers for TPUs, etc. "The cost today of rewriting your software to run on those stacks is very high... But the cost is going to go down." "You're going to have software that basically lets you take CUDA-compatible stuff and run it on alternative accelerators."

TBPN

35,189 Aufrufe • vor 3 Monaten

Chamath: US AI Startups Can Learn A Lot from DeepSeek "This is a case where necessity was the mother of invention." On E213, Chamath Palihapitiya explained how, even if the $6M number is inaccurate, DeepSeek still had some impressive breakthroughs: GRPO > PPO for reinforcement learning "These guys were like, 'Well, how am I going to do this reinforcement learning thing?' They invented a totally different algorithm." "It uses a lot less computer memory and it's highly performant." PTX > CUDA to build the model "And then the second thing that was crazy is everybody is used to building models and compiling through CUDA, which is NVIDIA's proprietary language." "And these guys worked totally around CUDA and they did something called PTX, which goes right to the bare metal." What US AI startups can learn from this: "We, meaning the West, with all the money that we've had, didn't come up with these ideas." "And I think part of why we didn't come up is not that we're not smart enough to do it, but we weren't forced to because the constraints didn't exist." "And so I just wonder how we make sure we learn this principle, meaning, when the AI company wakes up and rolls out of bed and some VC gives them $200M, maybe that's not the right answer for a Series A or a Seed." "And maybe the right answer is $2M so that they do these DeepSeek-like innovations."
0:55

Chamath: US AI Startups Can Learn A Lot from DeepSeek "This is a case where necessity was the mother of invention." On E213, Chamath Palihapitiya explained how, even if the $6M number is inaccurate, DeepSeek still had some impressive breakthroughs: GRPO > PPO for reinforcement learning "These guys were like, 'Well, how am I going to do this reinforcement learning thing?' They invented a totally different algorithm." "It uses a lot less computer memory and it's highly performant." PTX > CUDA to build the model "And then the second thing that was crazy is everybody is used to building models and compiling through CUDA, which is NVIDIA's proprietary language." "And these guys worked totally around CUDA and they did something called PTX, which goes right to the bare metal." What US AI startups can learn from this: "We, meaning the West, with all the money that we've had, didn't come up with these ideas." "And I think part of why we didn't come up is not that we're not smart enough to do it, but we weren't forced to because the constraints didn't exist." "And so I just wonder how we make sure we learn this principle, meaning, when the AI company wakes up and rolls out of bed and some VC gives them $200M, maybe that's not the right answer for a Series A or a Seed." "And maybe the right answer is $2M so that they do these DeepSeek-like innovations."

The All-In Podcast

108,815 Aufrufe • vor 1 Jahr

Can LLMs invent better ways to train LLMs? At Sakana AI, we’re pioneering AI-driven methods to automate AI research and discovery. We’re excited to release DiscoPOP: a new SOTA preference optimization algorithm that was discovered and written by an LLM! Our method leverages LLMs to propose and implement new preference optimization algorithms. We then train models with those algorithms and evaluate their performance, providing feedback to the LLM. By repeating this process for multiple generations in an evolutionary loop, the LLM discovers many highly-performant and novel preference optimization objectives! Paper: GitHub: Model: We proudly collaborated with the University of Oxford (Foerster Lab for AI Research (now part of BOLD)) and Cambridge University (Mihaela van der Schaar) on this groundbreaking project. Looking ahead, we envision a future where AI-driven research reduces the need for extensive human intervention and computational resources. This will accelerate scientific discoveries and innovation, pushing the boundaries of what AI can achieve.
0:22

Can LLMs invent better ways to train LLMs? At Sakana AI, we’re pioneering AI-driven methods to automate AI research and discovery. We’re excited to release DiscoPOP: a new SOTA preference optimization algorithm that was discovered and written by an LLM! Our method leverages LLMs to propose and implement new preference optimization algorithms. We then train models with those algorithms and evaluate their performance, providing feedback to the LLM. By repeating this process for multiple generations in an evolutionary loop, the LLM discovers many highly-performant and novel preference optimization objectives! Paper: GitHub: Model: We proudly collaborated with the University of Oxford (Foerster Lab for AI Research (now part of BOLD)) and Cambridge University (Mihaela van der Schaar) on this groundbreaking project. Looking ahead, we envision a future where AI-driven research reduces the need for extensive human intervention and computational resources. This will accelerate scientific discoveries and innovation, pushing the boundaries of what AI can achieve.

Sakana AI

555,922 Aufrufe • vor 2 Jahren

We gave our AI system a spec, and in under 2 weeks, it designed, verified, and deployed a chip that beats NVIDIA. It’s built for low-power physical AI workloads. We’re running live inference on >B+ parameter models like Llama, Qwen, and Kimi, serving at 3.4x better perf/watt than NVIDIA Jetson. From just a specification, our AI system autonomously generated all of the RTL Design, UVM verification, formal proofs, firmware, drivers, and kernels, co-designing the model, software and silicon as one optimization loop. Better AI can now design better chips to run AI, leading to a loop of recursive-self improvement towards our path to abundant intelligence.
1:04

We gave our AI system a spec, and in under 2 weeks, it designed, verified, and deployed a chip that beats NVIDIA. It’s built for low-power physical AI workloads. We’re running live inference on >B+ parameter models like Llama, Qwen, and Kimi, serving at 3.4x better perf/watt than NVIDIA Jetson. From just a specification, our AI system autonomously generated all of the RTL Design, UVM verification, formal proofs, firmware, drivers, and kernels, co-designing the model, software and silicon as one optimization loop. Better AI can now design better chips to run AI, leading to a loop of recursive-self improvement towards our path to abundant intelligence.

Architect Labs

618,854 Aufrufe • vor 1 Tag

World Labs CEO Dr. Fei-Fei Li says AI must change how we teach and evaluate students: "AI must change learning. AI must change K-16 learning." "The most precious resource of our entire world is human capital." "When we have gotten the technology that can answer standardized tests... when AI can do better than an average human, it's not about humans are bad. It's about we need to change the education system." "We need to change how we evaluate. We need to change the way we empower teachers to educate the next generation of students where they can use these tools, be empowered, and do things that we can never imagine." "All of the kids today should not be scared of AI. They should feel the human agency to lead AI, to use AI in the right way, and to use AI to make the impact that they want to make for the world." Fei-Fei Li at Bloomberg Tech live with Emily Chang
1:35

World Labs CEO Dr. Fei-Fei Li says AI must change how we teach and evaluate students: "AI must change learning. AI must change K-16 learning." "The most precious resource of our entire world is human capital." "When we have gotten the technology that can answer standardized tests... when AI can do better than an average human, it's not about humans are bad. It's about we need to change the education system." "We need to change how we evaluate. We need to change the way we empower teachers to educate the next generation of students where they can use these tools, be empowered, and do things that we can never imagine." "All of the kids today should not be scared of AI. They should feel the human agency to lead AI, to use AI in the right way, and to use AI to make the impact that they want to make for the world." Fei-Fei Li at Bloomberg Tech live with Emily Chang

a16z

81,845 Aufrufe • vor 2 Monaten

Ben Thompson explains how LLMs greatly diminished Nvidia's CUDA moat even as they sent the stock to the moon "So the weird thing about large language models is they were obviously incredible for Nvidia. That's why their stock went to the moon." "They have been on and off the most valuable company in the world." "It was also very bad for Nvidia. And the reason it was bad for Nvidia is that the play with CUDA is to build a developer ecosystem on top of CUDA." "But CUDA only works on Nvidia GPUs. So you get CUDA for free. It's easier to use, and it's a tremendous investment. Nvidia almost went under trying to build CUDA at a time when no one understood what they were doing or why they were wasting money on it." "And that's why Jensen Huang will get bristly, particularly when people question their rent-seeking or profit, whatever. It's like, no, they earned their spot fair and square." "Absolutely. It shouldn't be forgotten. They have earned every dollar they've gotten through 25 years of taking massive risks." "It bottomed out in October 2022. I wrote an article like three weeks before ChatGPT came out, tracing their bottoming-out history and their search for what was next." "'Nvidia in the Valley.' So, go back to this GTC. So I wrote an article at the time called 'Nvidia Waves and Moats'." "And what was interesting about that GTC was, number one, it was very boring. All the cool stuff kind of got scrubbed out." "Now, Jensen Huang has brought that stuff back, so the last few GTCs he's more talking about other things. Now it comes across as, oh, you're still looking for something beyond the LLM." "Because the problem with the LLM is it shifts the developer platform far above where Nvidia sits. All the activity is happening on top of LLMs. And so no one who's writing an AI application today is using CUDA." "Now, some people are, if you're training your own model and you're doing some low-level things or non-LLM things." "But the vast majority of the energy and all the money and the ecosystem is far removed from CUDA." "They have no idea and don't need to know or care what chips their application is running on. They're just on the OpenAI API, or the Anthropic API, or using Bedrock on Amazon, and it's sitting on Trainium, and they're using a Chinese open-source model. It's totally abstracted away, and this is why LLMs were bad for Nvidia." "Now, again, all the money they made along the way is worth it, but their moat has been tremendously diminished." "CUDA is still a moat if you need to do stuff that requires CUDA. But the vast majority of stuff, in energy, doesn't require CUDA, like in a post-LLM world."
3:04

Ben Thompson explains how LLMs greatly diminished Nvidia's CUDA moat even as they sent the stock to the moon "So the weird thing about large language models is they were obviously incredible for Nvidia. That's why their stock went to the moon." "They have been on and off the most valuable company in the world." "It was also very bad for Nvidia. And the reason it was bad for Nvidia is that the play with CUDA is to build a developer ecosystem on top of CUDA." "But CUDA only works on Nvidia GPUs. So you get CUDA for free. It's easier to use, and it's a tremendous investment. Nvidia almost went under trying to build CUDA at a time when no one understood what they were doing or why they were wasting money on it." "And that's why Jensen Huang will get bristly, particularly when people question their rent-seeking or profit, whatever. It's like, no, they earned their spot fair and square." "Absolutely. It shouldn't be forgotten. They have earned every dollar they've gotten through 25 years of taking massive risks." "It bottomed out in October 2022. I wrote an article like three weeks before ChatGPT came out, tracing their bottoming-out history and their search for what was next." "'Nvidia in the Valley.' So, go back to this GTC. So I wrote an article at the time called 'Nvidia Waves and Moats'." "And what was interesting about that GTC was, number one, it was very boring. All the cool stuff kind of got scrubbed out." "Now, Jensen Huang has brought that stuff back, so the last few GTCs he's more talking about other things. Now it comes across as, oh, you're still looking for something beyond the LLM." "Because the problem with the LLM is it shifts the developer platform far above where Nvidia sits. All the activity is happening on top of LLMs. And so no one who's writing an AI application today is using CUDA." "Now, some people are, if you're training your own model and you're doing some low-level things or non-LLM things." "But the vast majority of the energy and all the money and the ecosystem is far removed from CUDA." "They have no idea and don't need to know or care what chips their application is running on. They're just on the OpenAI API, or the Anthropic API, or using Bedrock on Amazon, and it's sitting on Trainium, and they're using a Chinese open-source model. It's totally abstracted away, and this is why LLMs were bad for Nvidia." "Now, again, all the money they made along the way is worth it, but their moat has been tremendously diminished." "CUDA is still a moat if you need to do stuff that requires CUDA. But the vast majority of stuff, in energy, doesn't require CUDA, like in a post-LLM world."

Fireside Alpha

12,980 Aufrufe • vor 6 Tagen

Modal engineer who teaches Full-Stack Deep Learning explained everything an AI engineer needs to know about GPUs in 20 minutes - better than $800 CUDA workshops. memory bandwidth -> SM occupancy -> kernel fusion -> arithmetic intensity -> know when you're compute-bound vs memory-bound. That loop is how you stop overpaying for GPUs you're barely using. VRAM math + roofline model + Modal serverless GPUs + kernel profiling - that's the stack. Watch and save it, then check your own GPU utilization tonight.
19:25

Modal engineer who teaches Full-Stack Deep Learning explained everything an AI engineer needs to know about GPUs in 20 minutes - better than $800 CUDA workshops. memory bandwidth -> SM occupancy -> kernel fusion -> arithmetic intensity -> know when you're compute-bound vs memory-bound. That loop is how you stop overpaying for GPUs you're barely using. VRAM math + roofline model + Modal serverless GPUs + kernel profiling - that's the stack. Watch and save it, then check your own GPU utilization tonight.

h100envy

18,442 Aufrufe • vor 1 Monat

Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference optimization bootcamps. request comes in -> check LMCache -> hit? load KV cache from CPU/SSD/remote -> skip prefill -> serve. That loop is why Bloomberg and other production stacks now push 300 terabytes of KV cache per week. LMCache + vLLM + CPU/SSD/remote storage + zero-copy CUDA kernels - that's the stack. Watch and save it, then wire the KV-offload into your inference stack.
34:46

Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference optimization bootcamps. request comes in -> check LMCache -> hit? load KV cache from CPU/SSD/remote -> skip prefill -> serve. That loop is why Bloomberg and other production stacks now push 300 terabytes of KV cache per week. LMCache + vLLM + CPU/SSD/remote storage + zero-copy CUDA kernels - that's the stack. Watch and save it, then wire the KV-offload into your inference stack.

h100envy

61,202 Aufrufe • vor 1 Monat

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)
1:07

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

Rohan Paul

443,019 Aufrufe • vor 1 Monat

We raised a $40M Series A to build the automated lab for agentic biology. Last week, we shared the work we did with Paul Jankura: Claude designed proteins, sent them to our automated wet lab, and got real experimental data back. We believe this is where biology is going: AI agents designing experiments, running them in automated wet labs, learning from the results and iterating. But AI can only move as fast as the experiments behind it. To use the potential of AI to cure all diseases, we need to build high-throughput wet lab infrastructure: a “biological gigafactory”. That’s exactly what we’re doing at Adaptyv. Over the past year we’ve grown our lab throughput by over 5x and onboarded more than 100 customers, ranging from bio AI labs like Chai Discovery and Boltz to pharmas like Roche and Novo Nordisk to dozens and dozens of new startups that use AI to radically speed up drug discovery.
0:25

We raised a $40M Series A to build the automated lab for agentic biology. Last week, we shared the work we did with Paul Jankura: Claude designed proteins, sent them to our automated wet lab, and got real experimental data back. We believe this is where biology is going: AI agents designing experiments, running them in automated wet labs, learning from the results and iterating. But AI can only move as fast as the experiments behind it. To use the potential of AI to cure all diseases, we need to build high-throughput wet lab infrastructure: a “biological gigafactory”. That’s exactly what we’re doing at Adaptyv. Over the past year we’ve grown our lab throughput by over 5x and onboarded more than 100 customers, ranging from bio AI labs like Chai Discovery and Boltz to pharmas like Roche and Novo Nordisk to dozens and dozens of new startups that use AI to radically speed up drug discovery.

Adaptyv Bio

163,997 Aufrufe • vor 3 Tagen