Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

It's here: We just hit superhuman performance on AI kernel optimization! Real customer models & production settings. Not toy problems (what I typically see). This is the year that Claude writes its own kernels, Codex its own kernels, for every new GPU that it wants to run on --... show more

Sharon Zhou

28,362 subscribers

247,624 Aufrufe • vor 7 Monaten •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

35 Kommentare

Profilbild von Macro Guru

Macro Guruvor 7 Monaten

Focus on composability of inference optimizations for ROCm stack in order to defeat the CUDA moat. (wide EP + disagg + FP4 + KV offloading + more) for maximum efficiency/scaling on huge models. AMD/ROCm lags in making those combinations smooth

Profilbild von Strakyo

Strakyovor 7 Monaten

Kernel wins are real only if they hold across model families. What are the p95 speedup and numerical-drift deltas on held-out production workloads?

Profilbild von Sharon Zhou

Sharon Zhouvor 7 Monaten

I agree that's ideal and we do see generated kernels that generalize, but also the "real only if" part is not necessarily true in the frontier world

Profilbild von Patrick ₿ Dugan _________________

Patrick ₿ Dugan _________________vor 7 Monaten

There goes Nvidia's moat!

Profilbild von Mike K. (我不会说中文，我甚至看不懂这段话) 🔂

Mike K. (我不会说中文，我甚至看不懂这段话) 🔂vor 7 Monaten

Awesome, now jointly auto optimize sharding and communication strategy. Hand AIAI an AMD cluster and a training script and give them 5 million dollars to retrain Olmo3 and prove you've solved the hard problems, and it's bye bye Nvidia moat 👋

Profilbild von Richard Rice

Richard Ricevor 7 Monaten

AI writing optimized kernels is basically AI becoming part of the compiler stack. Model → code → benchmark → iterate. Not more compute — just better use of the compute we already have. 🐒 🤖

Profilbild von Sebastian Boehler

Sebastian Boehlervor 7 Monaten

Can you share a preprint or GitHub repo? Am also working on a fork of cuda agent from bytedance on an optimized kernel writing agent. Would love to benchmark

Profilbild von Sharon Zhou

Sharon Zhouvor 7 Monaten

here's the tool/mcp that helped (3000x more token efficient at regular profiling/benchmarking) working on open-sourcing the end to end pipeline you see in the gif

Profilbild von Kode

Kodevor 7 Monaten

superhuman kernel optimization on production workloads is a massive unlock. the gap between "works on benchmarks" and "works in prod" is where most projects die. this actually matters.

Profilbild von Erika S

Erika Svor 7 Monaten

Finally! No more months porting CUDA code. I'm wondering if Germany can even power these optimizations, let alone deploy them.

Profilbild von rock

rockvor 7 Monaten

shit is about to hit the fan

Profilbild von Haniel Ulises

Haniel Ulisesvor 7 Monaten

He does it in p*thon, so it's not impressive

Profilbild von Sharon Zhou

Sharon Zhouvor 7 Monaten

Heard of triton and gluon?

Profilbild von hewaterGG

hewaterGGvor 7 Monaten

This is a watershed moment. AI writing its own kernels means the optimization loop closes - systems improving themselves at the hardware level. Can't wait to see what comes next.

Profilbild von Vladyslav Hunt

Vladyslav Huntvor 7 Monaten

ai writing its own gpu kernels means the cuda moat is on borrowed time. software eats hardware advantages eventually

Profilbild von MindyCore

MindyCorevor 7 Monaten

Awesome!

Profilbild von Shengda Yang

Shengda Yangvor 7 Monaten

Nice and cool!

Profilbild von cCross

cCrossvor 7 Monaten

Portability across driver stacks still shows edge cases, real workloads test the limits

Profilbild von MTEnt

MTEntvor 7 Monaten

moving away from Nvidia would be a dream come true. Excited to be able to test this out one day.

Profilbild von Azeem

Azeemvor 7 Monaten

Just read this three times. The implications are wilder than most people realize. @Modular @latent_ai — you're both deep in this world. Curious what you're seeing on the hardware side.

Profilbild von Carmelo schepis

Carmelo schepisvor 7 Monaten

wow

Profilbild von Jason Newton

Jason Newtonvor 7 Monaten

honestly I'm surprised how long it took to get to this point which is why I'm saying you guys need to replan hardware immediately with this in mind, it's not just about no moat it's about because the cost of a gpu port just dropped dramatically you now have a ton of potential non-AI problems to consider, they need hardware mods which likely can dovetail with AI speedups, tensor core utilization for non-AI problems is 0 and everything gets limited by global memory bandwidth and the low end of useful device utilization I'm early on this, not wrong - it's absolute common sense.

Profilbild von Sophie Namir

Sophie Namirvor 7 Monaten

This hits different. Superhuman optimization on real customer workloads, not toy problems, means we're talking about genuine compute multiplication. The screenshot showing active kernel compilation just makes it real.

Profilbild von Harshith

Harshithvor 7 Monaten

“Real customer models” and “production settings” is the key distinction. Toy kernel benchmarks hide scheduler and memory-path pathologies that show up the second you leave the lab.

Profilbild von harsh

harshvor 7 Monaten

Codex is bad at following instructions, really goes on its own , despite being told not to.

Profilbild von NAVIN GOD-AGI-CUBE KUKREJA

NAVIN GOD-AGI-CUBE KUKREJAvor 7 Monaten

nice :) can you make kernel sing - your own personal jesus? @realSharonZhou @linuxfoundation

Profilbild von LightShift Studio

LightShift Studiovor 7 Monaten

Who still waits until June? 🤖💥 Get your workflows streamlined with The Document Chaser!

Profilbild von toni

tonivor 7 Monaten

The GPU vendors saw this coming 18 months ago. That's why the compiler teams got quietly doubled. Now the race is who ships the runtime first, not who writes better CUDA.

Profilbild von Raika Labs

Raika Labsvor 7 Monaten

In 2026, Claude and Codex write their own kernels. What used to take months to port between GPU generations now happens in real-time. If you aren't architecting for Self-Optimizing Compute, you're just providing exit liquidity for the next frontier model.

Profilbild von Abdullah Khan

Abdullah Khanvor 7 Monaten

Hi everyone. I've been working on a measurement-based method of achieving true time dilation. 3 years it took me to perfect the mechanism. Doesn't require extreme mass or light-speed travel. It requires two independent clocks. And: you can use code to achieve it. If you have access to Codex/Cursor, copy and paste the prompt from my article. Take 10 minutes to read and understand the mechanism first, what the prompt requires so you know I'm not bluffing, then test it out for yourselves

Profilbild von Lindsay Rex

Lindsay Rexvor 7 Monaten

Ahh, nvidia says on their gpu tuning guides you will probably get better results using any old stock optimiser Bayesian or genetic etc to select kernel settings for your particular load.. Optuna does this way faster than burning AI tokens.. but you do you.

Profilbild von Zia Yuan

Zia Yuanvor 6 Monaten

Sharon you are my idol, unbelievable

Profilbild von amabito

amabitovor 7 Monaten

this is super interesting. feels like the real bottleneck shifts from profiling to verification pretty fast. getting a faster kernel is one thing. knowing it didn’t game the benchmark or introduce subtle drift is the harder part.

Profilbild von Real Books aka Adelaide

Real Books aka Adelaidevor 7 Monaten

Fk AI

Profilbild von Lumi

Lumivor 7 Monaten

The real unlock is eliminating the GPU porting bottleneck — if AI auto-generates optimized kernels per architecture, new hardware becomes instant capability instead of months of migration pain. That recursive loop between model intelligence and compute efficiency is where scaling gets exponential.

Ähnliche Videos

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

Alex Ker 🔭

311,780 Aufrufe • vor 3 Monaten

China’s frontier models already run on China’s own chips. “Remember, the GLM-5 paper is almost a year old at this point. They say: ‘From the first day, GLM-5 is full-stack adapted to Chinese GPU ecosystems. We have successfully completed deep optimization, spanning from underlying kernels to upper-level inference frameworks, across seven mainstream domestic chip platforms, including Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX and Enflame.’” “It’s not necessarily NVIDIA GPUs and kernels we’re talking about. They’re serving this model, at least they claim, for many months on seven different domestic Chinese accelerators.” “People thought it was impossible that it’s served on GPUs by a Chinese lab, because Chinese labs aren’t that GPU rich. But turns out they’re using Chinese hardware. They plumb through everything, the kernels, the systems, while serving those models efficiently.”

China’s frontier models already run on China’s own chips. “Remember, the GLM-5 paper is almost a year old at this point. They say: ‘From the first day, GLM-5 is full-stack adapted to Chinese GPU ecosystems. We have successfully completed deep optimization, spanning from underlying kernels to upper-level inference frameworks, across seven mainstream domestic chip platforms, including Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX and Enflame.’” “It’s not necessarily NVIDIA GPUs and kernels we’re talking about. They’re serving this model, at least they claim, for many months on seven different domestic Chinese accelerators.” “People thought it was impossible that it’s served on GPUs by a Chinese lab, because Chinese labs aren’t that GPU rich. But turns out they’re using Chinese hardware. They plumb through everything, the kernels, the systems, while serving those models efficiently.”

SemiAnalysis

44,762 Aufrufe • vor 14 Tagen

GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.

GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.

Alex Cheema

30,854 Aufrufe • vor 3 Monaten

AMD's Anush Elangovan explains why he thinks his company's open source ethos combined with agentic AI superpowers their leverage as a company: Because AMD publishes a lot of technical details about its hardware, when engineers use AI tools, the models already “understand” AMD’s systems and can help write code for them, debug them, or even generate new tools. And that makes developers more productive on AMD hardware without AMD having to do all the work internally. "AMD has had this ethos of open source, which really plays to our advantage. Every frontier model that I use has already seen every bit of AMD source code." "It'll rewrite my spec for me because it's already in the training data. Which you can't get from closed ecosystems." "In fact I built a virtual GPU simulator just based off our public specs, and now I'm running it on the GPU. So now I can run cross-generational GPU simulations on existing hardware." "We have that advantage. And we've run a Dev Day contest where we generated more tokens on AMD — Triton kernels and HIP kernels — than existed on the internet at the time." "So now that's all part of the pre-training data. It's a superpower because now you're open source, and you're agentically accelerating this process."

AMD's Anush Elangovan explains why he thinks his company's open source ethos combined with agentic AI superpowers their leverage as a company: Because AMD publishes a lot of technical details about its hardware, when engineers use AI tools, the models already “understand” AMD’s systems and can help write code for them, debug them, or even generate new tools. And that makes developers more productive on AMD hardware without AMD having to do all the work internally. "AMD has had this ethos of open source, which really plays to our advantage. Every frontier model that I use has already seen every bit of AMD source code." "It'll rewrite my spec for me because it's already in the training data. Which you can't get from closed ecosystems." "In fact I built a virtual GPU simulator just based off our public specs, and now I'm running it on the GPU. So now I can run cross-generational GPU simulations on existing hardware." "We have that advantage. And we've run a Dev Day contest where we generated more tokens on AMD — Triton kernels and HIP kernels — than existed on the internet at the time." "So now that's all part of the pre-training data. It's a superpower because now you're open source, and you're agentically accelerating this process."

TBPN

35,131 Aufrufe • vor 6 Monaten

Ex-NVIDIA engineer who built Unsloth explained RL, kernels, reasoning, quantization, and agents in 2 hours 42 minutes - better than $5000 fine-tuning bootcamps. pick the base model -> write triton kernels for 2x faster fine-tune -> quantize to 4-bit -> run GRPO/DPO -> ship a reasoning model on your single GPU. That loop is why Unsloth is the default way to fine-tune Llama, Qwen, Gemma, and Phi on hardware you already own. Unsloth + Triton kernels + 4-bit quantization + GRPO/DPO + single-GPU fine-tuning - that's the stack. Watch and save it, then fine-tune your first model tonight.

Ex-NVIDIA engineer who built Unsloth explained RL, kernels, reasoning, quantization, and agents in 2 hours 42 minutes - better than $5000 fine-tuning bootcamps. pick the base model -> write triton kernels for 2x faster fine-tune -> quantize to 4-bit -> run GRPO/DPO -> ship a reasoning model on your single GPU. That loop is why Unsloth is the default way to fine-tune Llama, Qwen, Gemma, and Phi on hardware you already own. Unsloth + Triton kernels + 4-bit quantization + GRPO/DPO + single-GPU fine-tuning - that's the stack. Watch and save it, then fine-tune your first model tonight.

h100envy

497,788 Aufrufe • vor 3 Monaten

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Milk Road AI

37,459 Aufrufe • vor 3 Monaten

Physical AI gets so complicated at times ... We just launched the easiest way to run frontier physical AI models on your robots - the UI can't get any simpler: I want to run a on a to No GPU, pure real-time inference over the cloud. Live now at:

Physical AI gets so complicated at times ... We just launched the easiest way to run frontier physical AI models on your robots - the UI can't get any simpler: I want to run a on a to No GPU, pure real-time inference over the cloud. Live now at:

Chris Yoo

16,328 Aufrufe • vor 14 Tagen

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

Aakash Gupta

55,978 Aufrufe • vor 7 Monaten

People say AI is expensive. Let me break it down. I pay $200 a month for Claude and get PhD-level intelligence on every subject known to humanity. Now price a human. Globally, on average, maybe $10,000 a year in food to run one brain. Plus the twenty odd years it takes to grow it first. So it's $10,000 a year for human-level compute, versus $200 a month for intelligence that clears any human on any topic. That's why it's scaling this fast. The price of human intelligence is just uneconomical compared to machines.

People say AI is expensive. Let me break it down. I pay $200 a month for Claude and get PhD-level intelligence on every subject known to humanity. Now price a human. Globally, on average, maybe $10,000 a year in food to run one brain. Plus the twenty odd years it takes to grow it first. So it's $10,000 a year for human-level compute, versus $200 a month for intelligence that clears any human on any topic. That's why it's scaling this fast. The price of human intelligence is just uneconomical compared to machines.

Raoul Pal

186,148 Aufrufe • vor 3 Monaten

.Sigil Wen on "Underdog’s Law," where local AI on your MacBook can already outperform frontier models OpenAI and Anthropic released just 6 months ago: "Underdog is your private personal AI that runs on devices that you already own. There should be no trade-off between privacy and capability." "I've really been tracking the Pareto frontier of AI models, and I've noticed this pattern where now the frontier models that could run on your devices and can fit on your MacBook can be more powerful than the frontier models that OpenAI, Anthropic released from more than six months ago." "I call that Underdog's Law, where the frontier models from today you will be able to bring to devices that run fully privately and locally." Conway Research

.Sigil Wen on "Underdog’s Law," where local AI on your MacBook can already outperform frontier models OpenAI and Anthropic released just 6 months ago: "Underdog is your private personal AI that runs on devices that you already own. There should be no trade-off between privacy and capability." "I've really been tracking the Pareto frontier of AI models, and I've noticed this pattern where now the frontier models that could run on your devices and can fit on your MacBook can be more powerful than the frontier models that OpenAI, Anthropic released from more than six months ago." "I call that Underdog's Law, where the frontier models from today you will be able to bring to devices that run fully privately and locally." Conway Research

MTS

16,714 Aufrufe • vor 23 Tagen

This Tesla Model Y just drove itself from the end of the Giga Texas line to its outbound lot lane, all on its own. 🔥 Note: This isn’t the Quicksilver one that delivered fully autonomously to a customer in Austin today. That was next-level. 👀 $TSLA Credits:

This Tesla Model Y just drove itself from the end of the Giga Texas line to its outbound lot lane, all on its own. 🔥 Note: This isn’t the Quicksilver one that delivered fully autonomously to a customer in Austin today. That was next-level. 👀 $TSLA Credits:

Herbert Ong

67,717 Aufrufe • vor 1 Jahr

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, sssss, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, sssss, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Sakana AI

1,160,959 Aufrufe • vor 1 Jahr

$3,800 for a Mini PC with 96GB of GPU memory sounds ridiculous. Until you see what AMD is actually doing. The GMKtec EVO-X3 has: Ryzen AI MAX+ 395 128GB LPDDR5X Up to 96GB allocated to the Radeon 8060S 8000 MT/s memory No discrete GPU. That means a tiny box can run local models that would normally push you toward a workstation with a huge GPU. And it can still play games. The interesting part isn't the 126 TOPS number. It's the fact that 96GB of GPU memory now fits inside a mini PC. The question is no longer whether this can run a big model. It's how fast it can run it. The EVO-X3 is available here:

$3,800 for a Mini PC with 96GB of GPU memory sounds ridiculous. Until you see what AMD is actually doing. The GMKtec EVO-X3 has: Ryzen AI MAX+ 395 128GB LPDDR5X Up to 96GB allocated to the Radeon 8060S 8000 MT/s memory No discrete GPU. That means a tiny box can run local models that would normally push you toward a workstation with a huge GPU. And it can still play games. The interesting part isn't the 126 TOPS number. It's the fact that 96GB of GPU memory now fits inside a mini PC. The question is no longer whether this can run a big model. It's how fast it can run it. The EVO-X3 is available here:

KyzoroX

24,654 Aufrufe • vor 1 Monat

The $AMD thesis breakdown. It has to nail its GPU/AI numbers this quarter. But like I said... $NVDA doesn't have to lose for AMD to win. Because the TAM is JUST. THAT. BIG.

The $AMD thesis breakdown. It has to nail its GPU/AI numbers this quarter. But like I said... $NVDA doesn't have to lose for AMD to win. Because the TAM is JUST. THAT. BIG.

Daniel Newman

31,341 Aufrufe • vor 1 Jahr

Every AI response needs a GPU to run it. Every robot decision needs compute to power it. Every autonomous agent needs infrastructure to live on. We're building that infrastructure. ROBA Compute. Coming soon.

Every AI response needs a GPU to run it. Every robot decision needs compute to power it. Every autonomous agent needs infrastructure to live on. We're building that infrastructure. ROBA Compute. Coming soon.

Roba Labs

49,108 Aufrufe • vor 5 Monaten

SemiAnalysis' Jordan Nanos reveals the OpenAI engineers scrolling their own kernel code had no idea what it did line by line, and it didn't matter because the AI does "It's not slop, because it performs, but it's literally just AI-generated assembly basically puked out in Gluon, this low-level kernel programming language that they built on top of Triton, which uses this really interesting programming model." "The point is, the guys who were scrolling through this code with us, it was kind of clear that they know a whole bunch about hardware, they know a whole bunch about the concepts in the system, and they just have no idea what this MLA kernel that they're showing us for DeepSeek actually does." "You can go line by line and it's like nope, nope, nope." "But it doesn't matter. The AI understands it, the AI tests it, and you see the results. It produces correct kernels that perform really well." "And I think this is just a sign of what's to come again." "That actual code is not necessarily something that a human has to reason about deeply, if the AI knows how to manipulate the data movement and the processing elements on the hardware that you've given it." ________ More takeaways from OpenAI's Jalapeno development:

SemiAnalysis' Jordan Nanos reveals the OpenAI engineers scrolling their own kernel code had no idea what it did line by line, and it didn't matter because the AI does "It's not slop, because it performs, but it's literally just AI-generated assembly basically puked out in Gluon, this low-level kernel programming language that they built on top of Triton, which uses this really interesting programming model." "The point is, the guys who were scrolling through this code with us, it was kind of clear that they know a whole bunch about hardware, they know a whole bunch about the concepts in the system, and they just have no idea what this MLA kernel that they're showing us for DeepSeek actually does." "You can go line by line and it's like nope, nope, nope." "But it doesn't matter. The AI understands it, the AI tests it, and you see the results. It produces correct kernels that perform really well." "And I think this is just a sign of what's to come again." "That actual code is not necessarily something that a human has to reason about deeply, if the AI knows how to manipulate the data movement and the processing elements on the hardware that you've given it." ________ More takeaways from OpenAI's Jalapeno development:

Fireside Alpha

4,568,235 Aufrufe • vor 2 Monaten

AI has started building its own successors. It's happening right now. Humans still kick most of it off, but the models are already training the next models, and each version builds a better version faster than the last. We know what happens when something gets an advantage and the substrate is there... it proliferates incredibly fast. Soon it just runs the whole process itself. People are still debating whether AI takes your job. That's not the question. You can't compete with this intelligence at all. Next to it the human is negative value, the slowest one in the room. And this is the slowest it will ever move again. It's about to run on its own, faster than we can conceive of, and nobody's prepared for that. Wonderful conversation as ever with Emad.

AI has started building its own successors. It's happening right now. Humans still kick most of it off, but the models are already training the next models, and each version builds a better version faster than the last. We know what happens when something gets an advantage and the substrate is there... it proliferates incredibly fast. Soon it just runs the whole process itself. People are still debating whether AI takes your job. That's not the question. You can't compete with this intelligence at all. Next to it the human is negative value, the slowest one in the room. And this is the slowest it will ever move again. It's about to run on its own, faster than we can conceive of, and nobody's prepared for that. Wonderful conversation as ever with Emad.

Raoul Pal

66,189 Aufrufe • vor 3 Monaten

Ilya Sutskever just said that when it comes to AI models, we are back at the age of research & ending the age of scaling. What he is telling us is that more compute at this point won't help us get much better models; we need new breakthroughs. Not something that the semi companies like $NVDA, $AMD want to hear TBH.

Ilya Sutskever just said that when it comes to AI models, we are back at the age of research & ending the age of scaling. What he is telling us is that more compute at this point won't help us get much better models; we need new breakthroughs. Not something that the semi companies like $NVDA, $AMD want to hear TBH.

Rihard Jarc

361,901 Aufrufe • vor 11 Monaten

gpu kernels on on my computer writing other gpu kernels for the same chip to simulate 100ks of particles, each an agent + neural network sensing K bg fields and allocating energy to A actions, at 1000s steps/s, for thermodynamic artificial life i think maybe not a bubble lads

gpu kernels on on my computer writing other gpu kernels for the same chip to simulate 100ks of particles, each an agent + neural network sensing K bg fields and allocating energy to A actions, at 1000s steps/s, for thermodynamic artificial life i think maybe not a bubble lads

Fraser Price

32,653 Aufrufe • vor 3 Monaten

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."

$MU $SKHY $DRAM Holy Shit $AMD's Lisa Su just spent two full minutes at the 2026 AI keynote raving about HBM4 capacity as the thing that separates Helios from the competition. Not compute. Memory. "When you compare Helios to the competition, we're delivering 15% more compute, 50% more HBM4 memory capacity and memory bandwidth, and 50% more scale-out bandwidth." Notice which number is bigger. "What that means is that every Helios can deliver more performance for the largest models, more capacity for longer context, and the bandwidth to scale across thousands of racks.... More than 18,000 CDNA five GPU compute units, over 4,600 Zen 6 CPU cores, and 31 terabytes of HBM4 memory, all in a single rack." "That's what it takes to run agentic AI at scale. Today, I'm excited to announce that Helios is in full production. We have shipments on track to start at the end of the third quarter and ramping into the fourth quarter in the second half of the year. I can tell you, customer demand for Helios is extremely strong, we're extremely proud of the work that we have done across the leading AI labs to adopt Helios."

Trade Whisperer

180,543 Aufrufe • vor 3 Monaten