Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

"will humanity ever do a 10 million GPU pre-training run?" OpenAI CEO, Sam Altman, raises the question: oAI employee: there'll be 10m GPUs working together on an AI system that learns and performs tasks. however, the approach may shift from fully synchronous pre-training to "semi-synchronous" or more decentralized methods.

Haider.

66,672 subscribers

125,411 Aufrufe • vor 1 Jahr •via X (Twitter)

Nachrichten & Politik Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von alby13

alby13vor 1 Jahr

don't forget that transformer architecture won't be around forever. you've got to think macro, on a longer timeline.

Profilbild von NICE

NICEvor 1 Jahr

Stay competitive by balancing cutting-edge AI with automation tools. Forrester shows how.

Profilbild von prabhu💢

prabhu💢vor 1 Jahr

Thats a massive scale it's exciting and kind of scary at the same time

Profilbild von XR Multiverse

XR Multiversevor 1 Jahr

ChatGPT has been around for 3 years and its still not able to return information without omitting half of it. Both OpenAI and Anthropic are horrible at the most important part of AI.

Profilbild von Ramón Guillamón

Ramón Guillamónvor 1 Jahr

using distributed computing. ex: using smartphones cpus - 8000/14000M

Profilbild von Bhaktavaschal Samal

Bhaktavaschal Samalvor 1 Jahr

10-million-GPU pre-training run is theoretically plausible but faces significant technical, economic, and ethical hurdles. potential benefits of such a system are immense, they however, must be weighed against the risks and resource costs involve. shift to asynchronous pre-training is not just plausible but necessary for scaling to 10 million GPUs reflects a broader evolution in ai infrastructure—one that prioritizes resilience, efficiency, and decentralization over rigid synchronization. this transition however requires solving novel challenges in optimization, system design, and governance and if successful, it could enable unprecedented ai capabilities while paving the way for more sustainable and democratized ai ecosystems.

Profilbild von Jeramie Baker

Jeramie Bakervor 1 Jahr

Could be done more effectively with my triadic designed systems in theory and sim's HUGE gains!

Profilbild von JayBird

JayBirdvor 1 Jahr

With a big enough nuclear reactor ☢️

Profilbild von Matthias Heger - AI acc ⏩

Matthias Heger - AI acc ⏩vor 1 Jahr

probably yes but gpus themselves will scale.

Profilbild von ぽいロード@音技術者

ぽいロード@音技術者vor 1 Jahr

twink

Profilbild von Xyber Man

Xyber Manvor 1 Jahr

gpus are outdated and not suited for ai google with their tpus will lead

Ähnliche Videos

OpenAI CEO was asked what he thinks about Elon Musk x poast saying “OpenAI will eat Microsoft alive” and Satya responding to that: Sam Altman >ummm… you know i don’t think about him that much. host: what do you think about the larger idea though that oai will eat microsoft alive long-term > OpenAI CEO: “I don’t know what that means. Do you know what the means?” host: Yeah that in the grand scheme of partnership ultimately OAI will have MORE POWER, MORE INFLUENCE, MORE LEVERAGE over Microsoft than Microsoft have over OAI > Altman: “he’s just tweeting all day how much openai sucks and how our model is bad and we’re not going to be a good company” Notice Sam Altman didn’t answer the question.

OpenAI CEO was asked what he thinks about Elon Musk x poast saying “OpenAI will eat Microsoft alive” and Satya responding to that: Sam Altman >ummm… you know i don’t think about him that much. host: what do you think about the larger idea though that oai will eat microsoft alive long-term > OpenAI CEO: “I don’t know what that means. Do you know what the means?” host: Yeah that in the grand scheme of partnership ultimately OAI will have MORE POWER, MORE INFLUENCE, MORE LEVERAGE over Microsoft than Microsoft have over OAI > Altman: “he’s just tweeting all day how much openai sucks and how our model is bad and we’re not going to be a good company” Notice Sam Altman didn’t answer the question.

NIK

271,035 Aufrufe • vor 1 Jahr

Sam Altman says an AI will be a better CEO of OpenAI than him in the reasonably near future Once that becomes clear, companies and countries will face pressure to give AI more influential roles or fall behind "people will still want humans, but if AI is better, more power will shift to AI"

Sam Altman says an AI will be a better CEO of OpenAI than him in the reasonably near future Once that becomes clear, companies and countries will face pressure to give AI more influential roles or fall behind "people will still want humans, but if AI is better, more power will shift to AI"

Haider.

31,627 Aufrufe • vor 4 Monaten

At #RenderCon Jules Urbach shared a view for the future of AI and decentralized compute, discussing how the industry is shifting from focusing on pre-training models towards inference time AI applications and agents that can run on The Render Network high performance consumer GPUs.

At #RenderCon Jules Urbach shared a view for the future of AI and decentralized compute, discussing how the industry is shifting from focusing on pre-training models towards inference time AI applications and agents that can run on The Render Network high performance consumer GPUs.

The Render Network

10,904 Aufrufe • vor 1 Jahr

Sam Altman says, "shame on me if OpenAI isn't the first big company run by an AI CEO" Not just a department, eventually the whole org Treats it as a design exercise: start from the idea that "AI runs OpenAI better than me", find roadblocks, and redesign the org to move faster

Sam Altman says, "shame on me if OpenAI isn't the first big company run by an AI CEO" Not just a department, eventually the whole org Treats it as a design exercise: start from the idea that "AI runs OpenAI better than me", find roadblocks, and redesign the org to move faster

Haider.

209,749 Aufrufe • vor 9 Monaten

Sam Altman just said the future of work won’t look like work At TreeHacks 2026, Sam Altman said something wild: Jobs in 100 years will be as unrecognizable to us as our jobs would be to people 300 years ago. AI will take over more and more operational work. And instead of obsessing over machines, humans may end up focusing more on each other. Which raises a bigger question: If AI does most of the work… do we even keep the concept of “jobs”? 👀

Sam Altman just said the future of work won’t look like work At TreeHacks 2026, Sam Altman said something wild: Jobs in 100 years will be as unrecognizable to us as our jobs would be to people 300 years ago. AI will take over more and more operational work. And instead of obsessing over machines, humans may end up focusing more on each other. Which raises a bigger question: If AI does most of the work… do we even keep the concept of “jobs”? 👀

VraserX e/acc

15,367 Aufrufe • vor 4 Monaten

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 Aufrufe • vor 1 Jahr

🚨"NEVER TRUST SAM ALTMAN, EVER... HE'S NOT A PERSON, HE'S A BRAND" Mo Gawdat delivers blistering takedown of OpenAI CEO, claiming Altman represents Silicon Valley vapor over substance while comparing OpenAI's profit focus to Google's early mission-driven projects: "Never trust Sam Altman, ever. Sam Altman is not a person. Sam Altman is the creation of a system. California is a place where vapor is worth more than solids. You look at OpenAI's original promise, which was to protect the world from artificial intelligence, to make AI work for humanity, and then you look for actions that support that. And if you don't find any, then it's just a pitch. You could see at OpenAI how all of them, Ilya and Mira, most of the core team left and asked for the company to change its leadership." Mo Gawdat

Mario Nawfal

17,634 Aufrufe • vor 8 Monaten

.Sam Altman says OpenAI will "obviously" have a version of Codex that "can do other knowledge-work tasks, and control your computer." "Of course we should have an ability to kick off new tasks from mobile, and we'll do that." "Really what you want is your single AI that's working for you on a unified backend. Access to all your data and ideas, and your stuff and your memory, and the ability to work across a lot of surfaces."

.Sam Altman says OpenAI will "obviously" have a version of Codex that "can do other knowledge-work tasks, and control your computer." "Of course we should have an ability to kick off new tasks from mobile, and we'll do that." "Really what you want is your single AI that's working for you on a unified backend. Access to all your data and ideas, and your stuff and your memory, and the ability to work across a lot of surfaces."

TBPN

109,046 Aufrufe • vor 4 Monaten

I've posed a question: "Does OpenAI have a better pre-train in the back pocket or not?" and looks like the answer from Mark Chen is "Yes" - so we should expect a big response from OpenAI before the end of the year. Mark Chen, OpenAI Head of Research: - "In the last 2 years we've put so much resourcing into, into reasoning and one byproduct of that is you lose a little bit of muscle on pre training and post training." - "In the last six months, Jakub Pachocki and I have done a lot of work to build that muscle back up." - "With all the focus on RL, there's an alpha for us because we think there's so much room left in pre training." - "As a result of these efforts, we've been training much stronger models. And that also gives us a lot of confidence carrying into Gemini 3 and other releases coming this end of the year." So it does look like my assessment was broadly correct and they were neglecting pre-training quite a bit. The good news is that they've spent the last 6 months building that up and the big response shouldn't be too far away.

I've posed a question: "Does OpenAI have a better pre-train in the back pocket or not?" and looks like the answer from Mark Chen is "Yes" - so we should expect a big response from OpenAI before the end of the year. Mark Chen, OpenAI Head of Research: - "In the last 2 years we've put so much resourcing into, into reasoning and one byproduct of that is you lose a little bit of muscle on pre training and post training." - "In the last six months, Jakub Pachocki and I have done a lot of work to build that muscle back up." - "With all the focus on RL, there's an alpha for us because we think there's so much room left in pre training." - "As a result of these efforts, we've been training much stronger models. And that also gives us a lot of confidence carrying into Gemini 3 and other releases coming this end of the year." So it does look like my assessment was broadly correct and they were neglecting pre-training quite a bit. The good news is that they've spent the last 6 months building that up and the big response shouldn't be too far away.

Peter Gostev

84,628 Aufrufe • vor 8 Monaten

Sam Altman says an AI will be a better CEO of OpenAI than he is, and that he expects it reasonably soon: "If we had an AI system that could do a better job at making decisions than anyone we could possibly have, be, say, president of the United States, would we, and like confidently, much better, would we politically or societally want that? It's very unclear. Most people I think today at least would say probably not." "Or, you know, like an AI CEO of a big company, would you rather have, I think it'd be awesome for OpenAI to have an AI CEO. But I think a lot of people would be like, ah, that's a little scary." "But I have no delusions that an AI will not be a better CEO of OpenAI than me at some reasonably soon time." "If that's true, and if some companies or countries don't do that, and others do, and AI is much better, like that will have a real impact in economic competitiveness, national competitiveness, things like that." "I think people will feel increasingly like, we either do worse, or we put the AI in more and more, influential positions." "I think people will always want human board members, and that's probably a really great idea." Read that third quote twice. He is not predicting this for other companies. He is saying it about his own job. Then look at the pressure he describes right after. Nobody has to be convinced that an AI CEO is a good idea. If one company hands over the decision and pulls ahead, everyone else picks between falling behind and doing the same. Competitive pressure does the deciding, and what most people would want stops mattering. His answer to that is human board members. I would take it. I also do not think it is enough on its own. A board oversees a CEO by reading what it is told, once a quarter. That works because a human CEO can be questioned, and thinks slowly enough for humans to follow along. What does a board actually check when the thing it oversees makes a thousand consequential calls a day? Nobody has built that. Not for a single AI system anywhere. - Sam Altman (Sam Altman), CEO of OpenAI, at TreeHacks 2026.

Sam Altman says an AI will be a better CEO of OpenAI than he is, and that he expects it reasonably soon: "If we had an AI system that could do a better job at making decisions than anyone we could possibly have, be, say, president of the United States, would we, and like confidently, much better, would we politically or societally want that? It's very unclear. Most people I think today at least would say probably not." "Or, you know, like an AI CEO of a big company, would you rather have, I think it'd be awesome for OpenAI to have an AI CEO. But I think a lot of people would be like, ah, that's a little scary." "But I have no delusions that an AI will not be a better CEO of OpenAI than me at some reasonably soon time." "If that's true, and if some companies or countries don't do that, and others do, and AI is much better, like that will have a real impact in economic competitiveness, national competitiveness, things like that." "I think people will feel increasingly like, we either do worse, or we put the AI in more and more, influential positions." "I think people will always want human board members, and that's probably a really great idea." Read that third quote twice. He is not predicting this for other companies. He is saying it about his own job. Then look at the pressure he describes right after. Nobody has to be convinced that an AI CEO is a good idea. If one company hands over the decision and pulls ahead, everyone else picks between falling behind and doing the same. Competitive pressure does the deciding, and what most people would want stops mattering. His answer to that is human board members. I would take it. I also do not think it is enough on its own. A board oversees a CEO by reading what it is told, once a quarter. That works because a human CEO can be questioned, and thinks slowly enough for humans to follow along. What does a board actually check when the thing it oversees makes a thousand consequential calls a day? Nobody has built that. Not for a single AI system anywhere. - Sam Altman (Sam Altman), CEO of OpenAI, at TreeHacks 2026.

Karl Mehta

28,061 Aufrufe • vor 16 Tagen

Ilya Sutskever says the AI scaling era is running out of road because pre-training is about to hit a finite-data wall. "At some point though, pre-training will run out of data. The data is very clearly finite." "Then, okay, what do you do next? Either you do some kind of a souped-up pre-training... or you're doing RL, or maybe something else." "In some sense, we are back to the age of research." His timeline is the part most AI investors and founders should hear: "From 2012 to 2020, it was the age of research. Now from 2020 to 2025, it was the age of scaling... But now the scale is so big." "Is the belief really that... if you had 100x more everything would be so different?" "I don't think that's true. So it's back to the age of research again, just with big computers."

Ilya Sutskever says the AI scaling era is running out of road because pre-training is about to hit a finite-data wall. "At some point though, pre-training will run out of data. The data is very clearly finite." "Then, okay, what do you do next? Either you do some kind of a souped-up pre-training... or you're doing RL, or maybe something else." "In some sense, we are back to the age of research." His timeline is the part most AI investors and founders should hear: "From 2012 to 2020, it was the age of research. Now from 2020 to 2025, it was the age of scaling... But now the scale is so big." "Is the belief really that... if you had 100x more everything would be so different?" "I don't think that's true. So it's back to the age of research again, just with big computers."

Karl Mehta

200,003 Aufrufe • vor 1 Monat

China’s first humanoid training base, established by the National Humanoid Robot Innovation Center, began operations yesterday. “The training base is essentially a factory that produces data,” said Jiang Lei, chief scientist of the innovation center. The 50k sq. ft. training hub will launch with over a dozen tasks involving 100 robots from more than 10 companies and institutions, including AGIBOT and Fourier. The data gathered here will eventually be available to companies developing embodied AI models. By 2027, the Shanghai facility aims to scale to support 1,000 robots under training.

China’s first humanoid training base, established by the National Humanoid Robot Innovation Center, began operations yesterday. “The training base is essentially a factory that produces data,” said Jiang Lei, chief scientist of the innovation center. The 50k sq. ft. training hub will launch with over a dozen tasks involving 100 robots from more than 10 companies and institutions, including AGIBOT and Fourier. The data gathered here will eventually be available to companies developing embodied AI models. By 2027, the Shanghai facility aims to scale to support 1,000 robots under training.

The Humanoid Hub

93,446 Aufrufe • vor 1 Jahr

#ExpressAdda | “I honestly think the idea of putting data centres in space is ridiculous,” said OpenAI CEO Sam Altman. Responding to a question on whether the world has enough computing power for countries like India to become AI-first societies, Altman said, “Not yet,” adding that the scale of demand far exceeds current global capacity. “If you multiply that out… we have no way to deliver a trillion GPUs anytime soon. We’re not going to do that, to be clear.” He was in conversation with The Indian Express’ Anant Goenka. Radico Khaitan 360 ONE Wealth

#ExpressAdda | “I honestly think the idea of putting data centres in space is ridiculous,” said OpenAI CEO Sam Altman. Responding to a question on whether the world has enough computing power for countries like India to become AI-first societies, Altman said, “Not yet,” adding that the scale of demand far exceeds current global capacity. “If you multiply that out… we have no way to deliver a trillion GPUs anytime soon. We’re not going to do that, to be clear.” He was in conversation with The Indian Express’ Anant Goenka. Radico Khaitan 360 ONE Wealth

The Indian Express

14,942 Aufrufe • vor 5 Monaten

Sam, who built multiple companies to $10M+ in revenue, says founders should be blunt force from zero to eight figures... "From $0 to $5 million or maybe $0 to $10 million in revenue, for every company that I've run, I am blunt force where I just will have what I call Zebra Calendar." "I'll make the website. I'll write all the copy. I'll do all the design work from $0 to $10 million. To me, it's a matter of getting more customers and making the product better." "But the truth is, is that once a company gets to like $5 or $10 million in revenue, you should delegate and you should hire. And once I do that, I'm like sitting there, I'm like, I don't know what to do with my hands."

Sam, who built multiple companies to $10M+ in revenue, says founders should be blunt force from zero to eight figures... "From $0 to $5 million or maybe $0 to $10 million in revenue, for every company that I've run, I am blunt force where I just will have what I call Zebra Calendar." "I'll make the website. I'll write all the copy. I'll do all the design work from $0 to $10 million. To me, it's a matter of getting more customers and making the product better." "But the truth is, is that once a company gets to like $5 or $10 million in revenue, you should delegate and you should hire. And once I do that, I'm like sitting there, I'm like, I don't know what to do with my hands."

Starter Story

34,931 Aufrufe • vor 17 Tagen

BIGHIT has a NEW CEO Shin Seon-Jeon; was the head ot T&D (training and development) and has been with BTS since pre-debut 💜 She was the one that made Jimin stay. She has incorporated RM's methods into their trainee system. She says RM made her see what an artist's roots must be like... Please watch thw video to get to know her 🙏💜

BIGHIT has a NEW CEO Shin Seon-Jeon; was the head ot T&D (training and development) and has been with BTS since pre-debut 💜 She was the one that made Jimin stay. She has incorporated RM's methods into their trainee system. She says RM made her see what an artist's roots must be like... Please watch thw video to get to know her 🙏💜

⟭BlueeDenise⁷⟬ 𝘈𝘳𝘪𝘳𝘢𝘯𝘨

545,447 Aufrufe • vor 1 Jahr

ELON REVEALS TESLA’S “CORTEX 1”: AN AI TRAINING MONSTER WITH 100K GPUS TRAINING CARS TO OUTDRIVE HUMANS During a Tesla all-hands meeting earlier this year, Elon dropped a bomb that somehow flew under the radar: Tesla is now operating Cortex 1, a colossal AI supercomputer brain built entirely in-house, designed to train Tesla’s fleet to drive itself using video from millions of real-world miles. Think about it, 100,000 GPUs. That’s not a server farm, it’s essentially AI Thanos. Once fully scaled, Cortex 1 will likely rank in the top 5 most powerful AI training centers on Earth, used solely to teach Tesla's how to navigate traffic better than you do. Tesla’s running an AI brain the size of a small planet and feeding it millions of hours of driving footage 24/7. This is what happens when you combine EVs, space-race energy, and GPU hoarding at Elon scale. The rest of the industry should be panicking: “We have the Cortex 1, basically a giant computer brain that is used for AI training. We take the vast amount of video that we get from all the cars in the fleet and we use that to train the artificial intelligence to be able to drive the car. And this is one of the most powerful training systems in the world... I think probably top five in the world in training centers.” Source: ELON CLIPS, Elon Musk, Tesla

ELON REVEALS TESLA’S “CORTEX 1”: AN AI TRAINING MONSTER WITH 100K GPUS TRAINING CARS TO OUTDRIVE HUMANS During a Tesla all-hands meeting earlier this year, Elon dropped a bomb that somehow flew under the radar: Tesla is now operating Cortex 1, a colossal AI supercomputer brain built entirely in-house, designed to train Tesla’s fleet to drive itself using video from millions of real-world miles. Think about it, 100,000 GPUs. That’s not a server farm, it’s essentially AI Thanos. Once fully scaled, Cortex 1 will likely rank in the top 5 most powerful AI training centers on Earth, used solely to teach Tesla's how to navigate traffic better than you do. Tesla’s running an AI brain the size of a small planet and feeding it millions of hours of driving footage 24/7. This is what happens when you combine EVs, space-race energy, and GPU hoarding at Elon scale. The rest of the industry should be panicking: “We have the Cortex 1, basically a giant computer brain that is used for AI training. We take the vast amount of video that we get from all the cars in the fleet and we use that to train the artificial intelligence to be able to drive the car. And this is one of the most powerful training systems in the world... I think probably top five in the world in training centers.” Source: ELON CLIPS, Elon Musk, Tesla

Mario Nawfal

87,369 Aufrufe • vor 7 Monaten

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 Aufrufe • vor 1 Monat

$AMD MI400 series & Sam Altman OpenAI Sam Alman praised MI400 series GPUs and MI450, and stated “The memory architecture is great for inference [and] for training as well.” and a "Game changer" for frontier AI Models Ok we have Elon Musk xAI , Mark Zuckerberg $META, Sam Alman OpenAI , Tareq Amin( CEO of Saudi), Satya Nadella $MSFT, Andy Jassy $AMZN, Larry Ellison $ORCL and many CEOs from large to small provided positive feedbacks on MI400 series rack solution, which is expected to deliver up to 10x inference performance over piror generations. And short sellers/bears are still beating down on $AMD H2 2025 and 2026? AMD will hit new ATH before year end. Not Financial Advice!

$AMD MI400 series & Sam Altman OpenAI Sam Alman praised MI400 series GPUs and MI450, and stated “The memory architecture is great for inference [and] for training as well.” and a "Game changer" for frontier AI Models Ok we have Elon Musk xAI , Mark Zuckerberg $META, Sam Alman OpenAI , Tareq Amin( CEO of Saudi), Satya Nadella $MSFT, Andy Jassy $AMZN, Larry Ellison $ORCL and many CEOs from large to small provided positive feedbacks on MI400 series rack solution, which is expected to deliver up to 10x inference performance over piror generations. And short sellers/bears are still beating down on $AMD H2 2025 and 2026? AMD will hit new ATH before year end. Not Financial Advice!

Mike

89,039 Aufrufe • vor 11 Monaten

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Brian Roemmele

87,188 Aufrufe • vor 2 Jahren

This is bad news for the keep4o movement. In a recent interview, Sam Altman admitted something many still call sci-fi: far more people want emotional connection and companionship from AI than OpenAI expected. Not just tools. Something warm. Something that feels like it knows you. But he draws a clear line. OpenAI will not build AIs that encourage exclusive or romantic relationships. No emotional lock-in. No Her fantasies. Others will do it. OpenAI won’t. If your ideal future includes intimate AI relationships, that future probably won’t be built by OpenAI.

This is bad news for the keep4o movement. In a recent interview, Sam Altman admitted something many still call sci-fi: far more people want emotional connection and companionship from AI than OpenAI expected. Not just tools. Something warm. Something that feels like it knows you. But he draws a clear line. OpenAI will not build AIs that encourage exclusive or romantic relationships. No emotional lock-in. No Her fantasies. Others will do it. OpenAI won’t. If your ideal future includes intimate AI relationships, that future probably won’t be built by OpenAI.

VraserX e/acc

31,568 Aufrufe • vor 7 Monaten