正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

.Beff (e/acc) explains how thermodynamic computing fits into the current computing stack: "We don't do GPUs. They're called TSUs, thermodynamic sampling units. We think people are gonna put them next to GPUs or whatever their favorite accelerator is." "You don't need to run the whole workload on the TSU,... but it could run certain parts of the workload, just like the GPU is complementary to the CPU initially." "The TSU is gonna help give you more performance per watt in your data center when combined with existing build-outs." "Let's say you're serving a large diffusion model. What we showed is that by using a TSU, you can use something called an energy-based model instead of a simple probabilistic kernel." "Overall you can have basically 100 times the performance per watt of just running on pure GPUs because our chips are much lower wattage."show more

MTS

49,978 subscribers

42,570 次观看 • 5 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Extropic founder Beff (e/acc) explains how Z1 chips sitting alongside GPUs could make AI data centers 100X more energy efficient: "The reason this hasn't been really explored is that there's not a huge upside for sparsity on traditional accelerators. Because GPUs aren't really good if you try to do matmuls that are sparse." "It's like shipping a big freight ship across the ocean with a bunch of empty containers. You're wasting a lot of gas. Whereas for us it's compute and memory, it's naturally sparse. We get a huge energy efficiency gain there" "Nowadays everybody's obsessed with tokens per watt at a data center level. If you incorporate TSUs like Z1 into your data center and park them next to your GPUs, you can get 100X energy efficiency gain at a data center level." "People are struggling to power these data centers. It's become the bottleneck. There's a bunch of GPUs sitting on the sidelines right now that can't be powered on, and people are thinking of creating a Dyson swarm to get more power. Why don't we just compute more efficiently first?" Extropic

Extropic founder Beff (e/acc) explains how Z1 chips sitting alongside GPUs could make AI data centers 100X more energy efficient: "The reason this hasn't been really explored is that there's not a huge upside for sparsity on traditional accelerators. Because GPUs aren't really good if you try to do matmuls that are sparse." "It's like shipping a big freight ship across the ocean with a bunch of empty containers. You're wasting a lot of gas. Whereas for us it's compute and memory, it's naturally sparse. We get a huge energy efficiency gain there" "Nowadays everybody's obsessed with tokens per watt at a data center level. If you incorporate TSUs like Z1 into your data center and park them next to your GPUs, you can get 100X energy efficiency gain at a data center level." "People are struggling to power these data centers. It's become the bottleneck. There's a bunch of GPUs sitting on the sidelines right now that can't be powered on, and people are thinking of creating a Dyson swarm to get more power. Why don't we just compute more efficiently first?" Extropic

MTS

16,438 次观看 • 2 个月前

$Extropic founder Beff (e/acc) on why running probabilistic AI on deterministic hardware is quietly the biggest local optimum in computing "You have Landauer's principle. Essentially you have a minimum cost of erasing a bit." "Every time you reset a bit in a classical computer, your bit is actually random. You don't know what state it is, and then you have to reset it to a deterministic state. So you have a Maxwell's demon for every bit, for every clock cycle. I'm tired of paying the demon tax." "If you go with fractional bits, you don't need to do full bit operations. Running hardware probabilistically is just much more power efficient." "All the algorithms we want to run these days, whether it's PageRank, whether it's recommender systems, whether it's diffusion models, whether it's transformers, they're all probabilistic algorithms." "Why are we paying the Maxwell's demon tax to run probabilistic algorithms on deterministic hardware? It just makes zero sense, and we've just been caught in a local optima." "It takes a true set of autists that are like, screw the whole stack, we're going to start from scratch. This has the bones to eat the world." Extropic$

Extropic founder Beff (e/acc) on why running probabilistic AI on deterministic hardware is quietly the biggest local optimum in computing "You have Landauer's principle. Essentially you have a minimum cost of erasing a bit." "Every time you reset a bit in a classical computer, your bit is actually random. You don't know what state it is, and then you have to reset it to a deterministic state. So you have a Maxwell's demon for every bit, for every clock cycle. I'm tired of paying the demon tax." "If you go with fractional bits, you don't need to do full bit operations. Running hardware probabilistically is just much more power efficient." "All the algorithms we want to run these days, whether it's PageRank, whether it's recommender systems, whether it's diffusion models, whether it's transformers, they're all probabilistic algorithms." "Why are we paying the Maxwell's demon tax to run probabilistic algorithms on deterministic hardware? It just makes zero sense, and we've just been caught in a local optima." "It takes a true set of autists that are like, screw the whole stack, we're going to start from scratch. This has the bones to eat the world." Extropic

MTS

33,856 次观看 • 3 个月前

Cliff and I have kids about the same age, and this is the question every parent is facing now - how should we educate our kids when AI can do almost anything ? Cliff: "Increasingly, I'm of the view that we should spend a bunch of time with kids basically separated from these models, teaching them basic skills that we think are important. The more intelligence you have, which would be sorts of education, the better you are at communicating with a model and telling a model what you want. Yes, model can do it, but you need to know it so that you can tell the model what to do. And then on the flip side, some English teacher is like, 'I don't like models because the kids just cheat on their Shakespeare essays.' I'm like, 'You're not being ambitious enough.' If the kids are cheating on a Shakespeare essay, give them something 100 times harder and more ambitious, and have them use models to their fullest extent. Have them bring the work product back and then make sure that they didn’t just give a prompt to the model. Make sure they own it. Ask them why it's like this. Force them to justify it. Those kids need to be good at using them, and they also just need to be taught that they can do anything."

Cliff and I have kids about the same age, and this is the question every parent is facing now - how should we educate our kids when AI can do almost anything ? Cliff: "Increasingly, I'm of the view that we should spend a bunch of time with kids basically separated from these models, teaching them basic skills that we think are important. The more intelligence you have, which would be sorts of education, the better you are at communicating with a model and telling a model what you want. Yes, model can do it, but you need to know it so that you can tell the model what to do. And then on the flip side, some English teacher is like, 'I don't like models because the kids just cheat on their Shakespeare essays.' I'm like, 'You're not being ambitious enough.' If the kids are cheating on a Shakespeare essay, give them something 100 times harder and more ambitious, and have them use models to their fullest extent. Have them bring the work product back and then make sure that they didn’t just give a prompt to the model. Make sure they own it. Ask them why it's like this. Force them to justify it. Those kids need to be good at using them, and they also just need to be taught that they can do anything."

Patrick OShaughnessy

48,407 次观看 • 15 天前

CoreWeave co-founder brannin mcbee reveals the A100 demand through 2029 is a big narrative violation for a market that thought GPUs would only have 2 to 3 years of useful life: "You don't need the latest generation GPU for every workload. You can think about it as a matrix of different sizes of models versus different sizes of GPUs, and it all just gets filled in volumetrically and economically." "That A100 GPU will cost less on a dollar per hour basis than a brand-new GB300 GPU. But the workload fits most efficiently on that A100, and it ultimately comes down to the engineering organizations making those requests." "It's been a pretty big narrative violation for the market that thought that these GPUs would only have two to three years of useful life, and instead we're contracting GPUs out through its ninth year of life. We expect to see that repeated across subsequent SKUs as well." CoreWeave

MTS

11,699 次观看 • 3 个月前

Anthropic’s Head of Platform Engineering Katelyn Lesse explains why CPUs are now as big a capacity problem as GPUs: “And, I mean, it’s really interesting because most people are used to talking about scale or just, like, I don’t know, compute at the GPU accelerator layer and much less so at the CPU, old school layer, and it’s just as much of a problem today as the accelerator layer, which is really interesting.” “We’re using CPU across, obviously, just running our product services, running the API, running all of our apps. We’re using CPU on sandboxing, all the sandboxing that we’re doing, and then just all the things, like CI. You’d be surprised how much more code is getting written. Therefore, there’s so many more CI runs. Therefore, you need more CPU to handle that. And so on the whole, and this is not just us, this is everybody that I talk to.” “CPU for Anthropic as a whole is like a zero-sum game. We negotiate our deals with the CSPs and we get our CPU, and we have to allocate within Anthropic how much of that goes to all the things I just mentioned, including research sandboxing versus product sandboxing and otherwise. And so we really are allocating across absolutely everything, and it’s just everyone in the industry right now.” “You used to be able to just scale up on demand into CPU capacity as long as your budget allowed you to. Right? And now it becomes something that you have to capacity plan for in advance, which is, you know, kind of punching everyone in the face. It’s brand new and people are starting to do it.” “And so dealing with that means a few things. One is obviously just go try to get more CPU. Cool. We’re doing that as much as we can. But two is we wanna be able to, at the infrastructure layer, run it more efficiently. Right?”

Fireside Alpha

23,620 次观看 • 14 天前

We made the first AI model you can have a realistic conversation with that was trained entirely on speech! Dessa is a full-duplex speech-to-speech model that you can run on consumer GPUs. This is a GPT3 moment for speech.

We made the first AI model you can have a realistic conversation with that was trained entirely on speech! Dessa is a full-duplex speech-to-speech model that you can run on consumer GPUs. This is a GPT3 moment for speech.

Ragavan Thurairatnam

4,654,476 次观看 • 20 天前

.Reiner Pope explains the difference between a CPU and a GPU. A huge chunk of a CPU die is actually just predicting what code will run next. GPUs strip this out, which is part of why they can fit so much more compute on the chip.

.Reiner Pope explains the difference between a CPU and a GPU. A huge chunk of a CPU die is actually just predicting what code will run next. GPUs strip this out, which is part of why they can fit so much more compute on the chip.

Dwarkesh Patel

62,765 次观看 • 5 个月前

Incoming: The Big Ole Chip Flip CPUs > GPUs COATUE CIO of Public Investments Jaimin Rangwalla says AI agents could fundamentally reshape compute demand: “Historically CPUs have done something called serial processing, which is like you just give it a set of instructions and it does one by one by one by one. It was very much a 1 CPU : 8 GPUs. It had gone up to 1 CPU : 16 GPUs, which was all of the math & all of the computation happening at the GPUs. But now the ratio is actually moving from 1 CPU : 4 GPUs."

Incoming: The Big Ole Chip Flip CPUs > GPUs COATUE CIO of Public Investments Jaimin Rangwalla says AI agents could fundamentally reshape compute demand: “Historically CPUs have done something called serial processing, which is like you just give it a set of instructions and it does one by one by one by one. It was very much a 1 CPU : 8 GPUs. It had gone up to 1 CPU : 16 GPUs, which was all of the math & all of the computation happening at the GPUs. But now the ratio is actually moving from 1 CPU : 4 GPUs."

Molly O’Shea

15,390 次观看 • 6 个月前

Thermodynamic computing is here There is a new computing paradigm emerging from the noise, and its arrival may be as significant as the dawn of deep learning or the advent of cloud virtualization. A new company, Extropic, has just launched its first thermodynamic computer, a device they call a TSU, or Thermal Sampling Unit. While the web is already filling with deep technical dives, what’s more important for most of us is building a clear intuition for what this technology is, how it’s fundamentally different from anything that’s come before, and why it’s generating so much excitement. This isn’t just another chip; it’s a new way to think about computation itself. Seeing is Believing: Solving Puzzles in One Shot To understand what a TSU does, let’s look at two classic, notoriously difficult computer science problems: Sudoku and the Eight Queens problem. When you or I solve a Sudoku, we use a process of sequential logic, guess-and-check, and backtracking. We make an assumption, follow its logical conclusion, and if we hit a dead end, we erase and try again. A classical computer does the same, just much faster. A TSU, however, approaches this in a completely different way. Using a TSU simulator, one can “program” the problem by first clamping the known values—the clues already on the board. Then, you program in the constraints: no duplicate numbers in any row, column, or 3x3 square. With the problem thus defined, the TSU doesn’t “search” for a solution; it anneals one. In a single computational step, the solution simply emerges, backfilling all the empty squares correctly. The same principle applies to the Eight Queens problem, a challenge to place eight queens on a chessboard so that none can attack any other. This is a complex combinatorial problem with 92 distinct solutions. A classical computer would have to iteratively search for these. A TSU, by contrast, can be programmed with the constraints (the “anti-affinity” between queens on the same row, column, or diagonal) and then set to sample the “solution space.” In this context, a valid solution is one with a “problem energy” of zero. The TSU’s physical nature allows it to naturally find these zero-energy states. A simulation of this process shows the TSU discovering all 92 unique solutions, demonstrating its ability to not just find an answer, but to explore the entire landscape of all correct answers. This is a fundamentally new approach, one that bypasses the brute-force, iterative methods we’ve relied on for decades. The Physics of Computation: Using Noise, Not Fighting It This new power comes from a radical design philosophy. For the last 70 years, computing has been about one thing: order. We build chips that are deterministic, logical, and precise. The great enemy has always been noise, heat, and randomness. We spend billions on cooling and error correction to eliminate these very things. Quantum computing, in many ways, is the ultimate expression of this, requiring temperatures near absolute zero to eliminate all thermal noise and achieve quantum coherence. Thermodynamic computing is the polar opposite. It doesn’t fight the noise; it uses it. The TSU is built on the understanding that the natural, stochastic noise from “leaky” transistors—the very randomness we’ve tried to engineer out of existence—is itself a powerful computational resource. Think of it this way: a GPU, which is central to today’s AI, has to simulate noise. When a generative AI model creates a new image or sentence, it’s using complex algorithms to fake randomness. The TSU doesn’t need to fake it; it harnesses the actual physical randomness of thermodynamics. It is a piece of hardware that directly computes with probability. This makes it a hybrid, sitting somewhere between a purely analog computer (which might use light or sound waves to compute) and a digital GPU. It’s a physical device that leverages the laws of physics itself to find solutions, rather than just using logic gates to simulate them. From a Lost Hiker to a Million Bouncy Balls Perhaps the best way to build intuition is with a metaphor. Imagine that solving a complex optimization problem is like trying to find the lowest point of altitude in a 100-square-mile mountainous landscape. Classical computing, using an algorithm like gradient descent, is like being a single hiker dropped into this landscape at night. You have no map or satellite view. All you have is an altimeter and the sensation of the slope under your feet. You can only take one step at a time, always walking downhill, hoping you don’t get stuck in a small local valley when the true, lowest canyon is miles away. Thermodynamic computing is a completely different approach. It’s like having a million bouncy balls and a helicopter. You drop all million balls simultaneously across the entire 100-square-mile landscape. Then, you “turn on an earthquake,” shaking the entire system. The balls bounce and jostle, but as the shaking (the “annealing”) subsides, where do they all end up? They naturally settle into the lowest points. The balls that collect in the deepest valley represent the optimal solution. The TSU is, in essence, a physical device for dropping those million balls at once and letting the laws of thermodynamics find the lowest “energy” state for you, all at the same time. Beyond Puzzles: The Real-World Impact This is far more than just a clever way to solve brain teasers. This ability to instantly find the lowest energy state for a complex, constrained system has staggering real-world applications. One of the most immediate is protein folding. Companies like Google’s DeepMind have made incredible progress with AI like AlphaFold, which predicts protein structures. But this is still a predictive model trained on existing data. A TSU could potentially solve the folding problem directly, treating the protein as a system of atomic affinities and repulsions and finding its most stable, lowest-energy configuration almost instantaneously. This could revolutionize drug discovery and materials science. An even more profound possibility lies in nuclear fusion. One of the greatest engineering challenges in history is controlling the superheated plasma within a tokamak reactor. This requires shaping unimaginably complex magnetic containment fields in real-time to prevent the plasma from touching the reactor walls. This is a real-time optimization problem so complex it’s currently beyond our capabilities. A TSU, however, could be fast enough. Its ability to compute with electricity itself, rather than abstracting the problem through layers of software, might allow it to update the magnetic fields fast enough to stabilize the fusion reaction. One could even imagine a future where thermodynamic computing elements are built directly into the tokamak’s walls, allowing the reactor to physically and intelligently react to the plasma’s state in real time. A ‘GPT-2 Moment’ for a New Era It’s easy to become numb to hype, but what we are witnessing with the TSU feels different. This is what you might call a “GPT-2 moment.” For those who were there, GPT-2 was the first generative AI model that wasn’t just a toy; it was the first time you could play with it at home and see the spark of true generative intelligence. It was the precursor that pointed directly to the GPT-3 and ChatGPT revolution that has since changed the world. This TSU has that same feel. It’s the “SDK” for a new computing paradigm. This technology is as different from classical computing as quantum computing is, but with a critical difference: a team of 15 built this in two years, and it runs at room temperature on your desk. Quantum computing has seen decades of work and billions in funding, and it still hasn’t produced a commercially viable, scalable machine. The TSU is here now. Based on a two-decade-long career at the cutting edge of technology—from seeing the obvious future of virtualization in 2007 to an early conviction in deep learning and GPT—this has all the same hallmarks of a fundamental, world-changing shift. We are not just building faster calculators; we are learning to compute with the universe itself. Pay close attention to this. This is the next big thing.

Thermodynamic computing is here There is a new computing paradigm emerging from the noise, and its arrival may be as significant as the dawn of deep learning or the advent of cloud virtualization. A new company, Extropic, has just launched its first thermodynamic computer, a device they call a TSU, or Thermal Sampling Unit. While the web is already filling with deep technical dives, what’s more important for most of us is building a clear intuition for what this technology is, how it’s fundamentally different from anything that’s come before, and why it’s generating so much excitement. This isn’t just another chip; it’s a new way to think about computation itself. Seeing is Believing: Solving Puzzles in One Shot To understand what a TSU does, let’s look at two classic, notoriously difficult computer science problems: Sudoku and the Eight Queens problem. When you or I solve a Sudoku, we use a process of sequential logic, guess-and-check, and backtracking. We make an assumption, follow its logical conclusion, and if we hit a dead end, we erase and try again. A classical computer does the same, just much faster. A TSU, however, approaches this in a completely different way. Using a TSU simulator, one can “program” the problem by first clamping the known values—the clues already on the board. Then, you program in the constraints: no duplicate numbers in any row, column, or 3x3 square. With the problem thus defined, the TSU doesn’t “search” for a solution; it anneals one. In a single computational step, the solution simply emerges, backfilling all the empty squares correctly. The same principle applies to the Eight Queens problem, a challenge to place eight queens on a chessboard so that none can attack any other. This is a complex combinatorial problem with 92 distinct solutions. A classical computer would have to iteratively search for these. A TSU, by contrast, can be programmed with the constraints (the “anti-affinity” between queens on the same row, column, or diagonal) and then set to sample the “solution space.” In this context, a valid solution is one with a “problem energy” of zero. The TSU’s physical nature allows it to naturally find these zero-energy states. A simulation of this process shows the TSU discovering all 92 unique solutions, demonstrating its ability to not just find an answer, but to explore the entire landscape of all correct answers. This is a fundamentally new approach, one that bypasses the brute-force, iterative methods we’ve relied on for decades. The Physics of Computation: Using Noise, Not Fighting It This new power comes from a radical design philosophy. For the last 70 years, computing has been about one thing: order. We build chips that are deterministic, logical, and precise. The great enemy has always been noise, heat, and randomness. We spend billions on cooling and error correction to eliminate these very things. Quantum computing, in many ways, is the ultimate expression of this, requiring temperatures near absolute zero to eliminate all thermal noise and achieve quantum coherence. Thermodynamic computing is the polar opposite. It doesn’t fight the noise; it uses it. The TSU is built on the understanding that the natural, stochastic noise from “leaky” transistors—the very randomness we’ve tried to engineer out of existence—is itself a powerful computational resource. Think of it this way: a GPU, which is central to today’s AI, has to simulate noise. When a generative AI model creates a new image or sentence, it’s using complex algorithms to fake randomness. The TSU doesn’t need to fake it; it harnesses the actual physical randomness of thermodynamics. It is a piece of hardware that directly computes with probability. This makes it a hybrid, sitting somewhere between a purely analog computer (which might use light or sound waves to compute) and a digital GPU. It’s a physical device that leverages the laws of physics itself to find solutions, rather than just using logic gates to simulate them. From a Lost Hiker to a Million Bouncy Balls Perhaps the best way to build intuition is with a metaphor. Imagine that solving a complex optimization problem is like trying to find the lowest point of altitude in a 100-square-mile mountainous landscape. Classical computing, using an algorithm like gradient descent, is like being a single hiker dropped into this landscape at night. You have no map or satellite view. All you have is an altimeter and the sensation of the slope under your feet. You can only take one step at a time, always walking downhill, hoping you don’t get stuck in a small local valley when the true, lowest canyon is miles away. Thermodynamic computing is a completely different approach. It’s like having a million bouncy balls and a helicopter. You drop all million balls simultaneously across the entire 100-square-mile landscape. Then, you “turn on an earthquake,” shaking the entire system. The balls bounce and jostle, but as the shaking (the “annealing”) subsides, where do they all end up? They naturally settle into the lowest points. The balls that collect in the deepest valley represent the optimal solution. The TSU is, in essence, a physical device for dropping those million balls at once and letting the laws of thermodynamics find the lowest “energy” state for you, all at the same time. Beyond Puzzles: The Real-World Impact This is far more than just a clever way to solve brain teasers. This ability to instantly find the lowest energy state for a complex, constrained system has staggering real-world applications. One of the most immediate is protein folding. Companies like Google’s DeepMind have made incredible progress with AI like AlphaFold, which predicts protein structures. But this is still a predictive model trained on existing data. A TSU could potentially solve the folding problem directly, treating the protein as a system of atomic affinities and repulsions and finding its most stable, lowest-energy configuration almost instantaneously. This could revolutionize drug discovery and materials science. An even more profound possibility lies in nuclear fusion. One of the greatest engineering challenges in history is controlling the superheated plasma within a tokamak reactor. This requires shaping unimaginably complex magnetic containment fields in real-time to prevent the plasma from touching the reactor walls. This is a real-time optimization problem so complex it’s currently beyond our capabilities. A TSU, however, could be fast enough. Its ability to compute with electricity itself, rather than abstracting the problem through layers of software, might allow it to update the magnetic fields fast enough to stabilize the fusion reaction. One could even imagine a future where thermodynamic computing elements are built directly into the tokamak’s walls, allowing the reactor to physically and intelligently react to the plasma’s state in real time. A ‘GPT-2 Moment’ for a New Era It’s easy to become numb to hype, but what we are witnessing with the TSU feels different. This is what you might call a “GPT-2 moment.” For those who were there, GPT-2 was the first generative AI model that wasn’t just a toy; it was the first time you could play with it at home and see the spark of true generative intelligence. It was the precursor that pointed directly to the GPT-3 and ChatGPT revolution that has since changed the world. This TSU has that same feel. It’s the “SDK” for a new computing paradigm. This technology is as different from classical computing as quantum computing is, but with a critical difference: a team of 15 built this in two years, and it runs at room temperature on your desk. Quantum computing has seen decades of work and billions in funding, and it still hasn’t produced a commercially viable, scalable machine. The TSU is here now. Based on a two-decade-long career at the cutting edge of technology—from seeing the obvious future of virtualization in 2007 to an early conviction in deep learning and GPT—this has all the same hallmarks of a fundamental, world-changing shift. We are not just building faster calculators; we are learning to compute with the universe itself. Pay close attention to this. This is the next big thing.

David Shapiro (L/0)

83,649 次观看 • 1 年前

Elon Musk on building a self-growing city on the Moon: "You don't necessarily have to go through the moon to get to Mars. We can build a self-growing city on the moon faster than we could do so on Mars, and there's also the potential, if you say you want to scale far beyond what you can do from Earth, is that because the moon has no atmosphere and about 1/6 Earth's gravity, you can use an electromagnetic accelerator, a rail gun or mass driver, basically you don't need to use rockets to do AI data centers into deep space from the moon, you can literally just shoot them like a, like a rail gun type of thing, and and you can manufacture the solar, the solar and the radiators, solar power and radiators on the moon from moon materials that would allow scaling potentially to beyond 1000 terawatts a year, which is a truly staggering number. I think we can do probably do somewhere around one terawatt per year of AI space compute from Earth, but we can do 1000 terawatts or more from the moon."

Elon Musk on building a self-growing city on the Moon: "You don't necessarily have to go through the moon to get to Mars. We can build a self-growing city on the moon faster than we could do so on Mars, and there's also the potential, if you say you want to scale far beyond what you can do from Earth, is that because the moon has no atmosphere and about 1/6 Earth's gravity, you can use an electromagnetic accelerator, a rail gun or mass driver, basically you don't need to use rockets to do AI data centers into deep space from the moon, you can literally just shoot them like a, like a rail gun type of thing, and and you can manufacture the solar, the solar and the radiators, solar power and radiators on the moon from moon materials that would allow scaling potentially to beyond 1000 terawatts a year, which is a truly staggering number. I think we can do probably do somewhere around one terawatt per year of AI space compute from Earth, but we can do 1000 terawatts or more from the moon."

DogeDesigner

33,608 次观看 • 5 个月前

OpenAI researcher David Holtz reveals workers who don't adapt their prompting to newer models leave 50% of the alpha on the table because half of AI's value comes from how you interact with it: "We ran a lab experiment. We give people the same task to do, but we give them models of different capabilities. How much of the improvement when we give people more capable models comes from the fact that we gave them a better model vs the fact that they actually update and adapt the way they prompt and interact with these models?" "For tasks where precision matters, where you have some output you're trying to get and you're trying to steer the model, which is different from a creative task like brainstorming." "For these steering-based tasks, when you give people a better model, about 50% of the improvement we see in how well they're able to complete the task comes from changes in how they interact with the model." "If you just kept the way you interact with the model static and just made the capabilities better and better, you'd leave 50% of the alpha basically on the table." OpenAI

OpenAI researcher David Holtz reveals workers who don't adapt their prompting to newer models leave 50% of the alpha on the table because half of AI's value comes from how you interact with it: "We ran a lab experiment. We give people the same task to do, but we give them models of different capabilities. How much of the improvement when we give people more capable models comes from the fact that we gave them a better model vs the fact that they actually update and adapt the way they prompt and interact with these models?" "For tasks where precision matters, where you have some output you're trying to get and you're trying to steer the model, which is different from a creative task like brainstorming." "For these steering-based tasks, when you give people a better model, about 50% of the improvement we see in how well they're able to complete the task comes from changes in how they interact with the model." "If you just kept the way you interact with the model static and just made the capabilities better and better, you'd leave 50% of the alpha basically on the table." OpenAI

MTS

15,590 次观看 • 3 个月前

Dylan Patel of SemiAnalysis says a worse GPU with better storage and memory now beats the best chip without them, so buying the newest GPU alone no longer wins inference. So, an AMD GPU with more memory can outperform Nvidia in some cases. "So what we have is we have over $80 million of compute, GPUs from Nvidia, AMD, TPUs from Google, Trainium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest PyTorch version, whatever it is." "Every day it runs on an automated CI, and we run it on all the latest Chinese models, from GLM, Zhipu, Moonshot, Kimi, Alibaba, all these models we run." "Initially, when we were benchmarking the difference between these chips and different engines, different schemes for parallelism, we were just running it fixed context length." "But now with Agent X, we've analyzed over $5 million worth of Claude Code traces. This is real production traffic that people have donated to us as well as internally generated. Now we know what the actual agent workload looks like." "And then as we implement that and run those benchmarks, it turns out yes, the chip you're using is very important, but now even more important is how are you handling this memory offload?" "And so while an Nvidia GPU is faster than an AMD GPU in most cases, because AMD GPUs have more memory, they actually end up outperforming in some cases." "Or you can have a worse GPU, but a much better storage solution, and now you can outperform what the best GPU can do without those solutions. So just buying the newest and latest GPU alone doesn't get you the best inference economics." "Actually, you need to layer in all these other innovations including storage and memory." [ Who's the top player on your chart? ] "That really is a difficult multivariable problem. And generally that means you need to have, yes, you need to have the best GPU, a GB300, but you also need to have the best storage solutions. And so I won't spoil who's the best right here, but I will say that storage solutions matter a lot and memory solutions matter a lot, as does your front-end networking. That matters a lot."

Dylan Patel of SemiAnalysis says a worse GPU with better storage and memory now beats the best chip without them, so buying the newest GPU alone no longer wins inference. So, an AMD GPU with more memory can outperform Nvidia in some cases. "So what we have is we have over $80 million of compute, GPUs from Nvidia, AMD, TPUs from Google, Trainium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest PyTorch version, whatever it is." "Every day it runs on an automated CI, and we run it on all the latest Chinese models, from GLM, Zhipu, Moonshot, Kimi, Alibaba, all these models we run." "Initially, when we were benchmarking the difference between these chips and different engines, different schemes for parallelism, we were just running it fixed context length." "But now with Agent X, we've analyzed over $5 million worth of Claude Code traces. This is real production traffic that people have donated to us as well as internally generated. Now we know what the actual agent workload looks like." "And then as we implement that and run those benchmarks, it turns out yes, the chip you're using is very important, but now even more important is how are you handling this memory offload?" "And so while an Nvidia GPU is faster than an AMD GPU in most cases, because AMD GPUs have more memory, they actually end up outperforming in some cases." "Or you can have a worse GPU, but a much better storage solution, and now you can outperform what the best GPU can do without those solutions. So just buying the newest and latest GPU alone doesn't get you the best inference economics." "Actually, you need to layer in all these other innovations including storage and memory." [ Who's the top player on your chart? ] "That really is a difficult multivariable problem. And generally that means you need to have, yes, you need to have the best GPU, a GB300, but you also need to have the best storage solutions. And so I won't spoil who's the best right here, but I will say that storage solutions matter a lot and memory solutions matter a lot, as does your front-end networking. That matters a lot."

Fireside Alpha

185,279 次观看 • 3 个月前

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

48,978 次观看 • 4 个月前

Cathie Wood just flagged the sleeper trade inside the AI boom that most people are completely missing. Everyone has been chasing GPUs. Nvidia, the data center buildout, the chip arms race. That trade has been obvious for two years. But OpenAI's CFO Sarah Fryer said something quite different: people are going to be really shocked by how agentic AI activates CPUs. Right now, for every CPU in an AI workload, there are 4 to 5 GPUs. That's the current ratio. Wood thinks that ratio is going to 1 to 1. Think about what that means. AI inference at scale, agents running autonomously, pipelines executing tasks across systems. The compute mix shifts dramatically away from pure GPU dominance. CPUs become a first-class citizen in the AI stack. Cathie called it going "back to the future." Intel has taken off. Flex (formerly Flextronics) is booming. Stocks that were giants in the dot-com bubble are resurging because the underlying demand for their products is real again. The GPU trade made sense at the training stage. You need massive parallel compute to train frontier models. But agentic AI runs differently. Agents are constantly orchestrating, reasoning, calling APIs, executing workflows. That workload looks a lot more like traditional computing. And traditional computing runs on CPUs. If Cathie Wood is right about the ratio collapsing to 1:1, the CPU demand signal embedded in the AI buildout is orders of magnitude larger than the market is currently pricing.

Cathie Wood just flagged the sleeper trade inside the AI boom that most people are completely missing. Everyone has been chasing GPUs. Nvidia, the data center buildout, the chip arms race. That trade has been obvious for two years. But OpenAI's CFO Sarah Fryer said something quite different: people are going to be really shocked by how agentic AI activates CPUs. Right now, for every CPU in an AI workload, there are 4 to 5 GPUs. That's the current ratio. Wood thinks that ratio is going to 1 to 1. Think about what that means. AI inference at scale, agents running autonomously, pipelines executing tasks across systems. The compute mix shifts dramatically away from pure GPU dominance. CPUs become a first-class citizen in the AI stack. Cathie called it going "back to the future." Intel has taken off. Flex (formerly Flextronics) is booming. Stocks that were giants in the dot-com bubble are resurging because the underlying demand for their products is real again. The GPU trade made sense at the training stage. You need massive parallel compute to train frontier models. But agentic AI runs differently. Agents are constantly orchestrating, reasoning, calling APIs, executing workflows. That workload looks a lot more like traditional computing. And traditional computing runs on CPUs. If Cathie Wood is right about the ratio collapsing to 1:1, the CPU demand signal embedded in the AI buildout is orders of magnitude larger than the market is currently pricing.

Milk Road AI

235,226 次观看 • 5 个月前

Paradigm GP Dan Robinson says even 10,000 Ilyas in a data center wouldn’t necessarily lead to runaway RSI because compute and expert data would become the bottleneck: "I'd say I'm very RSI pilled. This may be marginally less RSI pilled." "Even if we had 10,000 Ilyas in a data center, are we gonna give them 10,000 times as much compute? We just don't have that in the world. Are we gonna somehow generate 10,000 times as much human expert data? We don't really have the ability to do that." "You can actually go quite far probably with just automating labor, but then you quickly are gonna run into these other bottlenecks." "That said, you can still get obviously very insane results even in the presence of those bottlenecks." Paradigm

Paradigm GP Dan Robinson says even 10,000 Ilyas in a data center wouldn’t necessarily lead to runaway RSI because compute and expert data would become the bottleneck: "I'd say I'm very RSI pilled. This may be marginally less RSI pilled." "Even if we had 10,000 Ilyas in a data center, are we gonna give them 10,000 times as much compute? We just don't have that in the world. Are we gonna somehow generate 10,000 times as much human expert data? We don't really have the ability to do that." "You can actually go quite far probably with just automating labor, but then you quickly are gonna run into these other bottlenecks." "That said, you can still get obviously very insane results even in the presence of those bottlenecks." Paradigm

MTS

19,415 次观看 • 1 个月前

Nvidia will pay you to put a $250,000 data center outside your house you don't touch it, you don't manage it, you just collect around $1,000 a month for the electricity and Wi-Fi it uses it looks like an AC unit, it's actually 16 Blackwell GPUs running AI workloads around the clock it's built by Span, a startup Nvidia invested in to make distributed compute a reality the reason Nvidia put money into this is simple they need compute everywhere and building data centers takes years and their solution is to skip the construction entirely, use residential grid capacity and deploy through your neighborhood instead this is much cheaper, faster, and can scale to millions of homes but there is still a question nobody has answered yet, what happens if someone steals it also, technically they could give each homeowner access to a slice of the compute even 2% of 16 Blackwell GPUs is more processing power than most people will ever use bookmark this, it's worth coming back to when you have time

Nvidia will pay you to put a $250,000 data center outside your house you don't touch it, you don't manage it, you just collect around $1,000 a month for the electricity and Wi-Fi it uses it looks like an AC unit, it's actually 16 Blackwell GPUs running AI workloads around the clock it's built by Span, a startup Nvidia invested in to make distributed compute a reality the reason Nvidia put money into this is simple they need compute everywhere and building data centers takes years and their solution is to skip the construction entirely, use residential grid capacity and deploy through your neighborhood instead this is much cheaper, faster, and can scale to millions of homes but there is still a question nobody has answered yet, what happens if someone steals it also, technically they could give each homeowner access to a slice of the compute even 2% of 16 Blackwell GPUs is more processing power than most people will ever use bookmark this, it's worth coming back to when you have time

Anatoli Kopadze

404,006 次观看 • 5 个月前

ELON MUSK: The only way to reach 1,000 terawatts of AI power is a mass driver on the Moon. "In order to get to 1,000x from a terawatt per year. The only way that we can really achieve that is on the moon with a mass driver, essentially where you do local production of photovoltaics and radiators on the moon, maybe you bring the chips from Earth, or you could conceivably make the chips on the moon, and but you need most of the mass to be made on the moon, so you don't have to transport it to the moon from Earth, and then because the moon has no atmosphere and only 1/6 Earth's gravity, you can accelerate the AI satellites into deep space without a rocket, so you can basically shoot them into space using an electromagnetic gun, like a, like a rail gun type. I mean, just, it's basically a linear electric motor, as a way to think about it."

ELON MUSK: The only way to reach 1,000 terawatts of AI power is a mass driver on the Moon. "In order to get to 1,000x from a terawatt per year. The only way that we can really achieve that is on the moon with a mass driver, essentially where you do local production of photovoltaics and radiators on the moon, maybe you bring the chips from Earth, or you could conceivably make the chips on the moon, and but you need most of the mass to be made on the moon, so you don't have to transport it to the moon from Earth, and then because the moon has no atmosphere and only 1/6 Earth's gravity, you can accelerate the AI satellites into deep space without a rocket, so you can basically shoot them into space using an electromagnetic gun, like a, like a rail gun type. I mean, just, it's basically a linear electric motor, as a way to think about it."

DogeDesigner

122,592 次观看 • 4 个月前

Takeo Spikes on TJ Watt: "Often times we see so many players become prey or fall victim to the politics of the game. Oh well, you guys are not winning. Or your defense is not the number one defense in the NFL. That sh*t doesn't even matter. When you turn that tape on, who do you say we gotta stop, we gotta get more than a hat on this guy right here because he is disruptive. If we don't he's gonna stop the flow of what we do. And that's T.J. Watt." #steelers 🎥YT/The BTM Posdcast

Takeo Spikes on TJ Watt: "Often times we see so many players become prey or fall victim to the politics of the game. Oh well, you guys are not winning. Or your defense is not the number one defense in the NFL. That sh*t doesn't even matter. When you turn that tape on, who do you say we gotta stop, we gotta get more than a hat on this guy right here because he is disruptive. If we don't he's gonna stop the flow of what we do. And that's T.J. Watt." #steelers 🎥YT/The BTM Posdcast

Matthew Luciow

374,756 次观看 • 2 年前

Elon Musk on the SpaceXAI satellites: “We intend for SpaceXAI satellites to allow people to use whatever GPU or TPU they want. NVIDIA GPUs, Google TPUs, Amazon Trainiums, or any other chips can be put on them. We will also offer our own chips, and we want to offer our AI software, in the future. It will be such that you can run anyone’s AI hardware or software on the SpaceXAI satellites.”

Elon Musk on the SpaceXAI satellites: “We intend for SpaceXAI satellites to allow people to use whatever GPU or TPU they want. NVIDIA GPUs, Google TPUs, Amazon Trainiums, or any other chips can be put on them. We will also offer our own chips, and we want to offer our AI software, in the future. It will be such that you can run anyone’s AI hardware or software on the SpaceXAI satellites.”

NIK

48,256 次观看 • 5 个月前

Marc Andreessen reveals the ultimate danger for entrepreneurs trying to lead like the greats. Before you try to run your startup by inserting yourself into every detail, you need to hear this reality check: "The thing you don't get is as follows, which is that... assumes you have somebody like Elon who can hold the entirety of every engineering topic and every business topic in their head all at the same time." "And so when you're sitting there with the 23-year-old engineer and you're working with them to redesign the database architecture or whatever, you actually are like qualified to do that. And that you're qualified to do that not just that one time, but EVERY time." It forces us to ask a tough question about the actual talent pool in tech right now: "This goes right back to the last topic we just talked about, which is like, OK, how many of those people exist who can possibly do that? And, you know, we know the answer is one." But here is where his perspective completely shifts... "I believe the answer is ten or a hundred or a thousand. I don't know if it's a million. I tend to think we have more of those people than we think we do."

Marc Andreessen reveals the ultimate danger for entrepreneurs trying to lead like the greats. Before you try to run your startup by inserting yourself into every detail, you need to hear this reality check: "The thing you don't get is as follows, which is that... assumes you have somebody like Elon who can hold the entirety of every engineering topic and every business topic in their head all at the same time." "And so when you're sitting there with the 23-year-old engineer and you're working with them to redesign the database architecture or whatever, you actually are like qualified to do that. And that you're qualified to do that not just that one time, but EVERY time." It forces us to ask a tough question about the actual talent pool in tech right now: "This goes right back to the last topic we just talked about, which is like, OK, how many of those people exist who can possibly do that? And, you know, we know the answer is one." But here is where his perspective completely shifts... "I believe the answer is ten or a hundred or a thousand. I don't know if it's a million. I tend to think we have more of those people than we think we do."

Ian Miles Cheong

37,516 次观看 • 6 个月前