Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Hugging Face acquisition explained “A man making $120BN a year selling compute decides to buy a company that helps make compute more cost-effective so he can sell more compute. If end users have $1TN to spend on tokens, Nvidia would prefer that money flow through open-source people at 30%... show more

Harry Stebbings

674,331 subscribers

71,375 Aufrufe • vor 7 Tagen •via X (Twitter)

Wissenschaft & Technologie Finanzen Nachrichten & Politik

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Dylan on why Anthropic leading on models doesn't mean OpenAI loses. Even the tier 2 and 3 labs will sell out of tokens: "Everyone's like Anthropic won. They had Mythos, Opus 4.7 and their revenue is adding $10 billion a month all before OpenAI's alleged Spud release. So clearly Anthropic is in the lead and OpenAI is cooked. Anthropic has such bounds on compute, they can only grow it so fast. Dario used to gloat about how OpenAI was being too aggressive on compute and Anthropic was more sensible in their scaling. Now Anthropic is like I wish we had a lot more compute. OpenAI just raised a ton of money to get incremental compute in addition to the irresponsible levels of compute that they were buying. By the end of the year, let's say a 4.6 Opus tier model, the economy would spend $100 billion on. Anthropic won't have enough compute to do that. And presumably OpenAI and Google will hit that tier soon. Anthropic may get to charge 70%+ gross margins but if OpenAI hits it next, they charge 50% gross margins. They still get all of this incremental demand. There is such demand for these tokens and such limitations on compute. It's pretty clear even the tier 2 or tier 3 lab are going to be sold out of tokens. The economic value that the best model can deliver is growing faster than our ability to actually serve those tokens to people via the infrastructure."

Dylan on why Anthropic leading on models doesn't mean OpenAI loses. Even the tier 2 and 3 labs will sell out of tokens: "Everyone's like Anthropic won. They had Mythos, Opus 4.7 and their revenue is adding $10 billion a month all before OpenAI's alleged Spud release. So clearly Anthropic is in the lead and OpenAI is cooked. Anthropic has such bounds on compute, they can only grow it so fast. Dario used to gloat about how OpenAI was being too aggressive on compute and Anthropic was more sensible in their scaling. Now Anthropic is like I wish we had a lot more compute. OpenAI just raised a ton of money to get incremental compute in addition to the irresponsible levels of compute that they were buying. By the end of the year, let's say a 4.6 Opus tier model, the economy would spend $100 billion on. Anthropic won't have enough compute to do that. And presumably OpenAI and Google will hit that tier soon. Anthropic may get to charge 70%+ gross margins but if OpenAI hits it next, they charge 50% gross margins. They still get all of this incremental demand. There is such demand for these tokens and such limitations on compute. It's pretty clear even the tier 2 or tier 3 lab are going to be sold out of tokens. The economic value that the best model can deliver is growing faster than our ability to actually serve those tokens to people via the infrastructure."

Patrick OShaughnessy

295,307 Aufrufe • vor 4 Monaten

Nvidia $NVDA catches and passes China's open models by the end of 2027. That's Jason Calacanis @jason on All-In, and the mechanism is pricing. Jensen sells a company a full rack of compute, soup to nuts, total sovereignty, at 80 to 90% below what that company pays for tokens from OpenAI and Anthropic. He doesn't need margin on the tokens because he already made it on the hardware. Calacanis's read: starting now Nvidia is the number one competitor to OpenAI and Anthropic, not only for tokens but for enterprise compute, and the leading open source provider in America. What the rack-versus-tokens shift does to AI margins: Source: All-In Podcast -

Nvidia $NVDA catches and passes China's open models by the end of 2027. That's Jason Calacanis @jason on All-In, and the mechanism is pricing. Jensen sells a company a full rack of compute, soup to nuts, total sovereignty, at 80 to 90% below what that company pays for tokens from OpenAI and Anthropic. He doesn't need margin on the tokens because he already made it on the hardware. Calacanis's read: starting now Nvidia is the number one competitor to OpenAI and Anthropic, not only for tokens but for enterprise compute, and the leading open source provider in America. What the rack-versus-tokens shift does to AI margins: Source: All-In Podcast -

Podcast Alpha

26,746 Aufrufe • vor 6 Tagen

Ben Thompson says Nvidia is already cutting prices. The cut just doesn't show up in their margins: "Everyone's coming for them and they're still charging however much money for a chip. But they're not maintaining their margins, because who's buying? This circular financing people talk about, Nvidia's providing a 25% backstop. If you actually ascribe a value to Nvidia taking equity in the Neoclouds, they guarantee they're going to buy all their compute through 2030. And why do they do that? So the entity in question can get a lower cost of capital, so they can buy more GPUs. They get a lower cost of capital because Nvidia assumed the risk. Taking risk has a price. There's also a world where a ton of compute comes to market, the hyperscalers don't need this compute, and Nvidia's paying for compute that no one wants. That is a diminution of Nvidia's profitability if you look at their business holistically. What that is, is a price cut. The price just didn't show up in the margins."

Ben Thompson says Nvidia is already cutting prices. The cut just doesn't show up in their margins: "Everyone's coming for them and they're still charging however much money for a chip. But they're not maintaining their margins, because who's buying? This circular financing people talk about, Nvidia's providing a 25% backstop. If you actually ascribe a value to Nvidia taking equity in the Neoclouds, they guarantee they're going to buy all their compute through 2030. And why do they do that? So the entity in question can get a lower cost of capital, so they can buy more GPUs. They get a lower cost of capital because Nvidia assumed the risk. Taking risk has a price. There's also a world where a ton of compute comes to market, the hyperscalers don't need this compute, and Nvidia's paying for compute that no one wants. That is a diminution of Nvidia's profitability if you look at their business holistically. What that is, is a price cut. The price just didn't show up in the margins."

Patrick OShaughnessy

158,875 Aufrufe • vor 23 Tagen

Dylan Patel on Anthropic: "I think by Q3 they may even be profitable including SBC. Their margins on Opus 4.8 token is north of 80% for API price. They have a lot of deals where their total corporate gross margin gets clawed down a little bit because of how they do deals. They have the capability to pay, buying GPUs at above market rate. It is something other companies can't necessarily do. Every GPU I rent because I am out of compute capacity, I can immediately turn around and sell tokens on it or every TPU or Trainium at a positive margin. If I am running at 75% gross margins and I double the cost of compute, its fine I am still running at 50% gross margins. My NOI still goes up, so I will rent GPUs at almost whatever price I want to pay." Via Sequoia Capital

Dylan Patel on Anthropic: "I think by Q3 they may even be profitable including SBC. Their margins on Opus 4.8 token is north of 80% for API price. They have a lot of deals where their total corporate gross margin gets clawed down a little bit because of how they do deals. They have the capability to pay, buying GPUs at above market rate. It is something other companies can't necessarily do. Every GPU I rent because I am out of compute capacity, I can immediately turn around and sell tokens on it or every TPU or Trainium at a positive margin. If I am running at 75% gross margins and I double the cost of compute, its fine I am still running at 50% gross margins. My NOI still goes up, so I will rent GPUs at almost whatever price I want to pay." Via Sequoia Capital

P Equity Research 📰

73,740 Aufrufe • vor 2 Monaten

SITUATION EXPLAINED: Dwarkesh argues compute could get 10X more expensive. • Anthropic's revenue has been 10X-ing year over year while lab compute only 3X's • Three ways that gap can close: margins rise, compute gets more expensive, or labs shift compute to inference • All three are already happening, Anthropic went from 40% margins in 2025 to possibly 80%+ this year • But labs don't want the third one, heavy inference spend signals AI progress has stalled and you're now a cloud provider • Spot compute prices are up 40%+ since February, and labs pay well above spot for security and scale • Google is reportedly paying SpaceXAI $900 million a month for 110K GPUs, roughly 2X spot • Key claim: if a human-level software engineer ran on an H100, that H100 should rent for $250K a year, 15X today's price • The 3X annual compute growth is 1.4X Moore's Law, 1.2X new fabs, and 1.8X from AI taking wafer allocation from other devices • The fab piece is bottlenecked by EUV tool supply through 2030, and the wafer piece hits a wall by end of 2027 • If compute stays scarce, new labs need far more capital just to reach the same starting line • Compute gets cheap again only once robots can turn sand and copper into computers, which is gated on robotics, not RSI sof 𓋹: "This is what I'm most concerned about, a lack of innovation, not within companies, but a lack of new companies that can actually do something substantially new, because of the scarcity of compute." Theo Jaffee: "The biggest companies in the world by revenue are Amazon and Walmart, at 743 billion and 725 billion. If Anthropic makes 100 billion by the end of this year, that puts them at Target. To go from Target to bigger than Walmart in a year would be very impressive."

SITUATION EXPLAINED: Dwarkesh argues compute could get 10X more expensive. • Anthropic's revenue has been 10X-ing year over year while lab compute only 3X's • Three ways that gap can close: margins rise, compute gets more expensive, or labs shift compute to inference • All three are already happening, Anthropic went from 40% margins in 2025 to possibly 80%+ this year • But labs don't want the third one, heavy inference spend signals AI progress has stalled and you're now a cloud provider • Spot compute prices are up 40%+ since February, and labs pay well above spot for security and scale • Google is reportedly paying SpaceXAI $900 million a month for 110K GPUs, roughly 2X spot • Key claim: if a human-level software engineer ran on an H100, that H100 should rent for $250K a year, 15X today's price • The 3X annual compute growth is 1.4X Moore's Law, 1.2X new fabs, and 1.8X from AI taking wafer allocation from other devices • The fab piece is bottlenecked by EUV tool supply through 2030, and the wafer piece hits a wall by end of 2027 • If compute stays scarce, new labs need far more capital just to reach the same starting line • Compute gets cheap again only once robots can turn sand and copper into computers, which is gated on robotics, not RSI sof 𓋹: "This is what I'm most concerned about, a lack of innovation, not within companies, but a lack of new companies that can actually do something substantially new, because of the scarcity of compute." Theo Jaffee: "The biggest companies in the world by revenue are Amazon and Walmart, at 743 billion and 725 billion. If Anthropic makes 100 billion by the end of this year, that puts them at Target. To go from Target to bigger than Walmart in a year would be very impressive."

MTS

10,617 Aufrufe • vor 1 Monat

Jason Calacanis @jason on Nvidia: "I believe right now, starting today, Nvidia is the number one competitor for OpenAI and Anthropic, not just for tokens but for enterprise compute and I think now Jensen is the leading open source provider in America. I think he's going to take the world, he's going to catch up and blow past China and the Chinese models I think by the end of 2027. And then that means you're going to have a player...I am more convinced than ever now that we can win the race, because Jensen represents open source globally. He's going to be able to take a rack of compute, bring it into a company that is currently using tokens from OpenAI and Anthropic and say here's your rack, soup to nuts, complete vertical solution and you have total sovereignty and i'm reducing your prices 80 or 90% cause I don't need to make money off the tokens I made money off the hardware." Via The All-In Podcast $NVDA

Jason Calacanis @jason on Nvidia: "I believe right now, starting today, Nvidia is the number one competitor for OpenAI and Anthropic, not just for tokens but for enterprise compute and I think now Jensen is the leading open source provider in America. I think he's going to take the world, he's going to catch up and blow past China and the Chinese models I think by the end of 2027. And then that means you're going to have a player...I am more convinced than ever now that we can win the race, because Jensen represents open source globally. He's going to be able to take a rack of compute, bring it into a company that is currently using tokens from OpenAI and Anthropic and say here's your rack, soup to nuts, complete vertical solution and you have total sovereignty and i'm reducing your prices 80 or 90% cause I don't need to make money off the tokens I made money off the hardware." Via The All-In Podcast $NVDA

P Equity Research 📰

57,188 Aufrufe • vor 5 Tagen

Compute Wars: OpenAI vs Anthopic. Why was Opus 4.5 such a breakthrough? Anthropic got lots more compute from AWS Madison and New Carlisle sites likely more than doubling their capacity. This got Anthropic got close to OpenAI's total capacity, and probably much higher effective capacity available for new model runs. Remember that it takes 6+ months between getting capacity and releasing the model, so the extra OpenAI capacity might be aligning well with the 'spud' model rather than GPT-5.4. Unless something dramatic happens, OpenAI will pull away in terms of compute available in H2 2026, but 2027 will be close. Future years are less certain but so far OpenAI has much higher planned capacity, though can't imagine Anthropic isn't pushing as hard as they can to get lots more compute. Always watch the compute, other things matter, but any new capability breakthrough probably came from throwing more compute at it.

Compute Wars: OpenAI vs Anthopic. Why was Opus 4.5 such a breakthrough? Anthropic got lots more compute from AWS Madison and New Carlisle sites likely more than doubling their capacity. This got Anthropic got close to OpenAI's total capacity, and probably much higher effective capacity available for new model runs. Remember that it takes 6+ months between getting capacity and releasing the model, so the extra OpenAI capacity might be aligning well with the 'spud' model rather than GPT-5.4. Unless something dramatic happens, OpenAI will pull away in terms of compute available in H2 2026, but 2027 will be close. Future years are less certain but so far OpenAI has much higher planned capacity, though can't imagine Anthropic isn't pushing as hard as they can to get lots more compute. Always watch the compute, other things matter, but any new capability breakthrough probably came from throwing more compute at it.

Peter Gostev

158,276 Aufrufe • vor 5 Monaten

The open-source & open-weight frontier is 7 months behind the closed-model frontier. Let’s talk about it. Check out last week’s episode for the reasons why you might want to consider open-source & open-weight models. Welcome to the fourth episode of the Open Model Series with merve, Hugging Face AI/ML engineer & open-sourceress! We have been discussing open-source & open-weight AI: what it is, how it works, and how you can run them on your own compute. Please leave all your questions about open-source & open-weight AI in the comments! This is a super important topic that is novel to most people, and it can be challenging to grasp - so rest assured your questions are shared with many others!

The open-source & open-weight frontier is 7 months behind the closed-model frontier. Let’s talk about it. Check out last week’s episode for the reasons why you might want to consider open-source & open-weight models. Welcome to the fourth episode of the Open Model Series with merve, Hugging Face AI/ML engineer & open-sourceress! We have been discussing open-source & open-weight AI: what it is, how it works, and how you can run them on your own compute. Please leave all your questions about open-source & open-weight AI in the comments! This is a super important topic that is novel to most people, and it can be challenging to grasp - so rest assured your questions are shared with many others!

Harper Carroll

17,238 Aufrufe • vor 9 Tagen

There's an economics theorem called Alchian-Allen. And it has the very interesting implication that AI labs will be able to charge *higher* margins on their best models as compute gets scarcer. As compute gets more expensive, the cost of running any model goes up. So you might as well pay a bit more to make sure you're running the very best model. Which means the economics of being at the frontier improve, because if you’re not running the very best model, then you’re underutilizing this very precious compute. This pushes the AI model market towards winner-take-all; if you're the best, you can get away with charging an even higher margin. Dylan Patel tells me that we’re already seeing this today: all the revenue in the industry is on the best models. That’s the Alchian-Allen effect. If there’s a cost increase that’s roughly the same for all products, then the relative difference in price between higher and lower quality goods actually goes down. Consumers become relatively more willing to pay for the premium product. And it means that as the compute shortage hits, AI labs can capture more margin - not less, as you might expect - because consumers are choosing premium models more often.

There's an economics theorem called Alchian-Allen. And it has the very interesting implication that AI labs will be able to charge higher margins on their best models as compute gets scarcer. As compute gets more expensive, the cost of running any model goes up. So you might as well pay a bit more to make sure you're running the very best model. Which means the economics of being at the frontier improve, because if you’re not running the very best model, then you’re underutilizing this very precious compute. This pushes the AI model market towards winner-take-all; if you're the best, you can get away with charging an even higher margin. Dylan Patel tells me that we’re already seeing this today: all the revenue in the industry is on the best models. That’s the Alchian-Allen effect. If there’s a cost increase that’s roughly the same for all products, then the relative difference in price between higher and lower quality goods actually goes down. Consumers become relatively more willing to pay for the premium product. And it means that as the compute shortage hits, AI labs can capture more margin - not less, as you might expect - because consumers are choosing premium models more often.

Dwarkesh Patel

323,500 Aufrufe • vor 6 Monaten

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.

Karl Mehta

16,619 Aufrufe • vor 2 Monaten

Ben Thompson says Nvidia has already cut prices, they just show up as backstops and neocloud equity instead of margins "They're actually not maintaining their margins, because who is buying?" "This whole question of circular financing, people talk about Lucent and things like that. This whole deal and Nvidia's providing 25% backstop." "But if you actually ascribe a value to Nvidia's taking equity in the Neoclouds or whatever, they guarantee they're going to buy all their compute to 2030. And why do they do that? So that the entity in question can get a lower cost of capital, so they can buy more GPUs." "But implicit in that, why do they get a lower cost of capital? They get a lower cost of capital because Nvidia assumed risk." "This is my point before. Risk never disappears. It just appears somewhere else. Taking on risk has a price." "Now there is a world where AI takes off, it never stops, and everything is fine, and Nvidia captured all the upside of their risk." "But there's also a world where, say, this Neocloud they backed, a ton of compute comes to market, the hyperscalers have plenty of compute, they don't have enough demand, Nvidia is paying for a computer that no one wants. They just lost a bunch of money." "So if you think about it, there's an expected value of that investment. It's not zero. It's not 100%. It's somewhere in the middle." "But that is a diminution of Nvidia's profitability. If you actually look at their business holistically, what that is is a price cut."

Ben Thompson says Nvidia has already cut prices, they just show up as backstops and neocloud equity instead of margins "They're actually not maintaining their margins, because who is buying?" "This whole question of circular financing, people talk about Lucent and things like that. This whole deal and Nvidia's providing 25% backstop." "But if you actually ascribe a value to Nvidia's taking equity in the Neoclouds or whatever, they guarantee they're going to buy all their compute to 2030. And why do they do that? So that the entity in question can get a lower cost of capital, so they can buy more GPUs." "But implicit in that, why do they get a lower cost of capital? They get a lower cost of capital because Nvidia assumed risk." "This is my point before. Risk never disappears. It just appears somewhere else. Taking on risk has a price." "Now there is a world where AI takes off, it never stops, and everything is fine, and Nvidia captured all the upside of their risk." "But there's also a world where, say, this Neocloud they backed, a ton of compute comes to market, the hyperscalers have plenty of compute, they don't have enough demand, Nvidia is paying for a computer that no one wants. They just lost a bunch of money." "So if you think about it, there's an expected value of that investment. It's not zero. It's not 100%. It's somewhere in the middle." "But that is a diminution of Nvidia's profitability. If you actually look at their business holistically, what that is is a price cut."

Fireside Alpha

56,613 Aufrufe • vor 23 Tagen

Baseten's Charlie O'Neill on why the gap to the frontier is just compute and data: "Despite OpenAI and Anthropic and the closed source labs having more compute and more data, the recipe is the same. We're all scaling the same recipe here, and we're at slightly different points of the scaling curve. It's not like OpenAI or Anthropic have any secret sauce that differentiates them scaling up their multi-trillion parameter models from the rest of the open source ecosystem." "It really is just a matter of taking the core underlying architecture and training recipe, and throwing more compute and data at it, which is really promising." "Of course, these closed source labs probably have a long tail of optimizations they've made, which we're gonna continue to try and figure out in the open source community. But the core mechanism is there." "K3 is a great example of, well, if you do scale this recipe up, we're gonna get a pretty good model that's very, very close to the frontier, if not frontier itself." Charlie O'Neill Baseten

Baseten's Charlie O'Neill on why the gap to the frontier is just compute and data: "Despite OpenAI and Anthropic and the closed source labs having more compute and more data, the recipe is the same. We're all scaling the same recipe here, and we're at slightly different points of the scaling curve. It's not like OpenAI or Anthropic have any secret sauce that differentiates them scaling up their multi-trillion parameter models from the rest of the open source ecosystem." "It really is just a matter of taking the core underlying architecture and training recipe, and throwing more compute and data at it, which is really promising." "Of course, these closed source labs probably have a long tail of optimizations they've made, which we're gonna continue to try and figure out in the open source community. But the core mechanism is there." "K3 is a great example of, well, if you do scale this recipe up, we're gonna get a pretty good model that's very, very close to the frontier, if not frontier itself." Charlie O'Neill Baseten

MTS

17,404 Aufrufe • vor 1 Monat

Palantir CEO Alex Karp says enterprises want to "own the means of production" instead of "transferring their alpha" to OpenAI or Anthropic: "Why are [LLMs] charging for tokens if it's so valuable?" "If it was so valuable—let's say I can make you a billion dollars tomorrow. Wouldn't I say, 'I'll make you a billion dollars, and I want 30%?'" "Look at our financials. The reason why everyone is chillaxing with bad financials and growth while losing money, is the client refuses to pay the true cost." "The two places that actually make money—profit, free cash flow—are our application layer called ontology, and compute." "We can get the frontier application to be exactly the same as a frontier model without the risk of transferring the alpha of your business to another." Via CNBC

Palantir CEO Alex Karp says enterprises want to "own the means of production" instead of "transferring their alpha" to OpenAI or Anthropic: "Why are [LLMs] charging for tokens if it's so valuable?" "If it was so valuable—let's say I can make you a billion dollars tomorrow. Wouldn't I say, 'I'll make you a billion dollars, and I want 30%?'" "Look at our financials. The reason why everyone is chillaxing with bad financials and growth while losing money, is the client refuses to pay the true cost." "The two places that actually make money—profit, free cash flow—are our application layer called ontology, and compute." "We can get the frontier application to be exactly the same as a frontier model without the risk of transferring the alpha of your business to another." Via CNBC

Jawwwn

1,174,107 Aufrufe • vor 2 Monaten

Adaption Labs CEO Sara Hooker says data centers in space are "pretty bonkers", and explains what the current consensus gets wrong: "Most co-located hardware is pretty much for training. I think that's why you care [about Space], otherwise you can distribute. Inference compute, which is where everything's moving towards, you can spread that compute more easily. You can have multiple data centers [on Earth]." "So if you care about space, you probably only care about training compute. I think people underestimate the amount of failures that happen, and you don't want to get your training job interrupted." "The real issue is that GPUs still have failure rates. The 2% of GPUs that are just considered done every year, that's really your cost. It's how quickly you can replace those [in space] and what that looks like."

Adaption Labs CEO Sara Hooker says data centers in space are "pretty bonkers", and explains what the current consensus gets wrong: "Most co-located hardware is pretty much for training. I think that's why you care [about Space], otherwise you can distribute. Inference compute, which is where everything's moving towards, you can spread that compute more easily. You can have multiple data centers [on Earth]." "So if you care about space, you probably only care about training compute. I think people underestimate the amount of failures that happen, and you don't want to get your training job interrupted." "The real issue is that GPUs still have failure rates. The 2% of GPUs that are just considered done every year, that's really your cost. It's how quickly you can replace those [in space] and what that looks like."

TBPN

37,463 Aufrufe • vor 7 Monaten

What does it mean when an AI model is “open weights” and how is it different than “open source”? Welcome to the next episode of the Open Model Series with merve, Hugging Face AI/ML engineer & open-sourceress! Every week we will discuss open-source & open-weight AI: what it is, how it works, and how you can run them on your own compute. Episode 3 out next week; episode 1 linked below. Please leave all your questions about open-source & open-weight AI in the comments. This is a super important topic that is novel to most people, and it can be challenging to grasp - so your questions are shared with many others.

What does it mean when an AI model is “open weights” and how is it different than “open source”? Welcome to the next episode of the Open Model Series with merve, Hugging Face AI/ML engineer & open-sourceress! Every week we will discuss open-source & open-weight AI: what it is, how it works, and how you can run them on your own compute. Episode 3 out next week; episode 1 linked below. Please leave all your questions about open-source & open-weight AI in the comments. This is a super important topic that is novel to most people, and it can be challenging to grasp - so your questions are shared with many others.

Harper Carroll

32,624 Aufrufe • vor 24 Tagen

.Haseeb ＞|＜ on why beating North Korea's hackers now comes down to who can afford the most compute: "If the attackers are spending 500 bucks, and good guys are spending maybe a couple bucks, they're like, oh, I just wanna check and see if there's any low-hanging fruit. Otherwise I'm not gonna use this code if I can tell it's obviously busted. So they spend a couple dollars worth of compute, and they say, ah, it looks good enough. It's a well-funded project. It's open source. Yeah, I guess I'll use it." "But North Korea is spending 500 bucks, 1,000 bucks, 2,000 bucks just grinding and grinding and grinding, running multiple agents overnight trying to find an attack." "That means that no matter how many good people are spending $2 worth of GLM compute on you, they will never be able to find the depth of the tree that North Korea is searching for. None of the normal users will ever get there, which means it's only really on the company." "The company is the only party that can coordinate enough compute to be able to actually expand the search space enough to out search an attacker." "But it does mean that attackers have the same problem. So if North Korea and Russia and China are all trying to hack your protocol, but they all spend, let's say, $5,000 each, if you spend $10,000 you will actually find everything that they can find and more." "Which in that sense, it benefits defenders over attackers in the long run, because the company can actually outspend any individual attacker in principle, and that used to be not sufficient defense in the old model because of the fact that attackers were uncorrelated with each other." Dragonfly ＞|＜

.Haseeb ＞|＜ on why beating North Korea's hackers now comes down to who can afford the most compute: "If the attackers are spending 500 bucks, and good guys are spending maybe a couple bucks, they're like, oh, I just wanna check and see if there's any low-hanging fruit. Otherwise I'm not gonna use this code if I can tell it's obviously busted. So they spend a couple dollars worth of compute, and they say, ah, it looks good enough. It's a well-funded project. It's open source. Yeah, I guess I'll use it." "But North Korea is spending 500 bucks, 1,000 bucks, 2,000 bucks just grinding and grinding and grinding, running multiple agents overnight trying to find an attack." "That means that no matter how many good people are spending $2 worth of GLM compute on you, they will never be able to find the depth of the tree that North Korea is searching for. None of the normal users will ever get there, which means it's only really on the company." "The company is the only party that can coordinate enough compute to be able to actually expand the search space enough to out search an attacker." "But it does mean that attackers have the same problem. So if North Korea and Russia and China are all trying to hack your protocol, but they all spend, let's say, $5,000 each, if you spend $10,000 you will actually find everything that they can find and more." "Which in that sense, it benefits defenders over attackers in the long run, because the company can actually outspend any individual attacker in principle, and that used to be not sufficient defense in the old model because of the fact that attackers were uncorrelated with each other." Dragonfly ＞|＜

MTS

35,600 Aufrufe • vor 1 Monat

I think I figured out how SpaceX / Elon Musk are going to pay for the ~8GW of DC capacity SA has forecasted them to deploy next year… as Brad Gerstner mentioned in this clip, $NVDA shareholders do not want NVIDIA backstopping $100’s of billions of dollars worth of compute- it will terrify the market AND, it seems highly unlikely that the Hypers will pay directly for that much compute either since it will balloon their spending next year + at the economics needed here, it would all be going to Frontier labs (hurting the diversification strategy they all are trying to pursue) So… what I think is likely to happen: Brad Gerstner laid out that in the past, the financial structure of compute infra was you get 25% payback per year, you recoup the infra cost by end of year 4, then year 5 profit & year 6 is the kicker to bump IRR up nicely That’s now changed as the demand has become so intense at the Frontier labs that the payback period on infra has compressed maybe into <1yr basically, this $300-$400B in CapEx is going to show up in 3-6 month term lease agreements between SpaceX & Anthropic / OpenAI with enough margin to pay off the compute portion entirely (or vast majority) over the lifespan of the agreement money will trade hands basically up front from ANT/OAI to SpaceX then to NVIDIA. NVIDIA will be paid in full for the GPUs up front and they’ll commit to supporting Elon / SpaceX to stand up the compute in the timeframe specified (SpaceX will ultimately be on the hook for timelines) The kicker would be if NVIDIA captures any durable rent here for facilitating this… a perpetual dividend or something like that

I think I figured out how SpaceX / Elon Musk are going to pay for the ~8GW of DC capacity SA has forecasted them to deploy next year… as Brad Gerstner mentioned in this clip, $NVDA shareholders do not want NVIDIA backstopping $100’s of billions of dollars worth of compute- it will terrify the market AND, it seems highly unlikely that the Hypers will pay directly for that much compute either since it will balloon their spending next year + at the economics needed here, it would all be going to Frontier labs (hurting the diversification strategy they all are trying to pursue) So… what I think is likely to happen: Brad Gerstner laid out that in the past, the financial structure of compute infra was you get 25% payback per year, you recoup the infra cost by end of year 4, then year 5 profit & year 6 is the kicker to bump IRR up nicely That’s now changed as the demand has become so intense at the Frontier labs that the payback period on infra has compressed maybe into <1yr basically, this $300-$400B in CapEx is going to show up in 3-6 month term lease agreements between SpaceX & Anthropic / OpenAI with enough margin to pay off the compute portion entirely (or vast majority) over the lifespan of the agreement money will trade hands basically up front from ANT/OAI to SpaceX then to NVIDIA. NVIDIA will be paid in full for the GPUs up front and they’ll commit to supporting Elon / SpaceX to stand up the compute in the timeframe specified (SpaceX will ultimately be on the hook for timelines) The kicker would be if NVIDIA captures any durable rent here for facilitating this… a perpetual dividend or something like that

Nick Dorsey

113,333 Aufrufe • vor 1 Monat

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Dylan Patel of SemiAnalysis says speed is the moat in AI inference, and getting there runs through new compute (Groq/Cerebras), KV-cache offload, and networking optimization (Aria). "Most of the infrastructure is pretty optimized for high throughput, I would say, but more and more people want to pay for speed. It turns out agents are very sequential, task-oriented, and models running in sequential tasks." "Trying something, verifying it, checking if it's right, and you do that in a loop. It ends up being very slow." "And speed is the moat. The faster you go, speed is the moat." "So a lot of the workloads and high-end use cases that people are developing now are all focused on high interactivity, and the infrastructure there is quite different." "So let's break that down into a few components." "One is you can do new compute: you can do Groq or Cerebras-style compute, which is very much focused on high interactivity." "But even with existing GPUs you can do a few things. One, if you're focused heavily on time to first token, you might offload KV cache to storage, keep it cached, move it back into the compute whenever you need it, much faster than recomputing." "And the third thing you can do is you have to optimize the network."

Fireside Alpha

284,032 Aufrufe • vor 1 Monat