Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

OpenAI and Broadcom have developed a custom artificial intelligence chip called Jalapeno. OpenAI is now testing the samples. The companies say this chip can cut costs by 50%. Ed Ludlow reports

18,790 Aufrufe • vor 4 Monaten •via X (Twitter)

3 Kommentare

Profilbild von Pranav
Pranavvor 4 Monaten

50% cheaper inference compounds. inference is the cost you pay every day a model is serving users, not a one-time training bill. halve it and every product margin at openai just moved.

Profilbild von Inflectiv AI ⧉
Inflectiv AI ⧉vor 4 Monaten

🔥🔥

Profilbild von Dr. Paul De Santis, PharmD
Dr. Paul De Santis, PharmDvor 4 Monaten

Worst branding Award 🥇

Ähnliche Videos

Dylan Patel says Anthropic aims to build its chip in half the time OpenAI took, and explains how, without chip designers’ margins, a gigawatt costs about $20 billion, not $60 billion “I mean, that’s certainly the trend, is all of these companies are trying to make chips, right? OpenAI showed, I think for the first time ever in the industry, that you can make a first-generation chip that’s really good. They let us into the lab. We tested on our open-source InferenceX benchmark, running all these open-source Chinese models, and the performance was better than Blackwell. And that’s a first-generation chip, and they utilized AI heavily in the design and programming process." "And Anthropic is starting their chip program, and they want to do it in half the time that OpenAI did. So they want to get a chip. I mean, these are aggressive goals, but if models are able to help them, you know, really accelerate this process. You’ve got Google selling TPUs externally, you’ve got Amazon’s Trainium, really quite good now and fully usable in PyTorch and such, Meta’s very aggressive roadmap, Microsoft, et cetera." "If you look at the, you know, I said it went from $40 billion to build a gigawatt to 60. If you remove all the margin out of the chip design houses, right, the NVIDIA and Broadcom, Marvell, et cetera, you’re actually at like $20 billion a gigawatt." "And so, if you were able to do that, then you could pay for five gigawatts with one gigawatt worth of compute running inference. And so obviously the economics here are very aligned. And as Stacy said, these companies just have to run like crazy, right? You can’t afford to be 2x performance per dollar per year, because you’re going to get completely smashed by these other companies, right?” _______ Link and breakdown of the full panel with Dylan Patel and Stacy Rasgon (free):

Fireside Alpha

56,093 Aufrufe • vor 13 Tagen