Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Great to be back at another stellar Modular developer conference: #ModCon2026! Thanks to tim for leading a deeply engaging discussion on AI infrastructure, the economics of training and inference, and the looming massive impact of open models. Takeaway: Efficient open models can achieve 90% of frontier performance at 10%... show more

Dave Munichiello 🇺🇸

3,851 subscribers

11,661 views • 23 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The most effective AI systems don't rely on a single model. Frontier models provide state-of-the-art performance for complex tasks, while routers automatically select lightweight, open-source models for simpler jobs to optimize accuracy, latency, and cost. Learn more:

The most effective AI systems don't rely on a single model. Frontier models provide state-of-the-art performance for complex tasks, while routers automatically select lightweight, open-source models for simpler jobs to optimize accuracy, latency, and cost. Learn more:

NVIDIA AI

12,152 views • 6 months ago

Congrats to our partners Together AI on their $800M Series C fundraising round. Few teams have done more to make open models fast, accessible, and production-ready. We’re excited to keep working with Vipul Ved Prakash and the team on the next frontier of inference: lower costs, better performance, and open models at real scale.

Congrats to our partners Together AI on their $800M Series C fundraising round. Few teams have done more to make open models fast, accessible, and production-ready. We’re excited to keep working with Vipul Ved Prakash and the team on the next frontier of inference: lower costs, better performance, and open models at real scale.

Pearl Research Labs

14,131 views • 2 months ago

"Most enterprise workflows don't require frontier AI models. Outside of coding, tasks like summarization, document generation, and briefing can often be handled effectively by lower-cost open-source models. The real value is knowing when you need frontier performance and when cheaper models are more than good enough." Michael Mignano What percent of enterprise workflows requires frontier models today Nikesh Arora Matan Grinberg Aaron Katz Lin Qiao

"Most enterprise workflows don't require frontier AI models. Outside of coding, tasks like summarization, document generation, and briefing can often be handled effectively by lower-cost open-source models. The real value is knowing when you need frontier performance and when cheaper models are more than good enough." Michael Mignano What percent of enterprise workflows requires frontier models today Nikesh Arora Matan Grinberg Aaron Katz Lin Qiao

Harry Stebbings

36,428 views • 2 months ago

Gavin Baker Gavin Baker says open source is "awesome" for AI infrastructure names because it shifts the economic value from the frontier labs to the infrastructure, and it is already 80% plus of all tokens processed. "A misconception that a lot of people have is that open source models are somehow bad for AI. They're awesome for the AI infrastructure providers. They just shift economic value from the margins of the frontier labs to the infrastructure." "But I do think there's still a role for these frontier models. And it may be true, to date, frontier tokens are capturing 90% of the economic value, and open source tokens are probably 80% plus of tokens processed, and those ratios may be here to stay." _______ Follow Fireside Alpha for more highlights of the best business and technology conversations.

Gavin Baker Gavin Baker says open source is "awesome" for AI infrastructure names because it shifts the economic value from the frontier labs to the infrastructure, and it is already 80% plus of all tokens processed. "A misconception that a lot of people have is that open source models are somehow bad for AI. They're awesome for the AI infrastructure providers. They just shift economic value from the margins of the frontier labs to the infrastructure." "But I do think there's still a role for these frontier models. And it may be true, to date, frontier tokens are capturing 90% of the economic value, and open source tokens are probably 80% plus of tokens processed, and those ratios may be here to stay." _______ Follow Fireside Alpha for more highlights of the best business and technology conversations.

Fireside Alpha

162,832 views • 1 month ago

“AI MODELS COULD BECOME 10× SMALLER WITHOUT LOSING MOST OF THEIR INTELLIGENCE.” Mark Zuckerberg says one of the most powerful developments in AI is model distillation: taking the capabilities of a much larger model and transferring them into a much smaller one. - Meta's huge frontier models can serve as "teachers" for smaller models. - Zuckerberg says a model could potentially retain around 90–95% of the larger model's intelligence while being only ~10% of its size. - That means dramatically lower inference costs, memory requirements and computing power. - Developers can distill and customize these capabilities for their own applications. And that's where open-source models come into the picture.

“AI MODELS COULD BECOME 10× SMALLER WITHOUT LOSING MOST OF THEIR INTELLIGENCE.” Mark Zuckerberg says one of the most powerful developments in AI is model distillation: taking the capabilities of a much larger model and transferring them into a much smaller one. - Meta's huge frontier models can serve as "teachers" for smaller models. - Zuckerberg says a model could potentially retain around 90–95% of the larger model's intelligence while being only ~10% of its size. - That means dramatically lower inference costs, memory requirements and computing power. - Developers can distill and customize these capabilities for their own applications. And that's where open-source models come into the picture.

SciTech Era

77,374 views • 1 month ago

learned a lot from this conversation with Simon Mo and Matt Bornstein. biggest takeaways for me: -there are a lot of reasons why we should like open-weight models. a lot of these arguments stop at handwavy things like "what if the labs stop releasing frontier models to the public" or "it's lower cost." but simon's position as lead maintainer of vLLM and CEO of Inferact give him authority to talk about some of the other, more interesting and concrete reasons to pay attention to open-weight models, namely that they allow end-users to calibrate latency / other performance metrics with way more customizability than what any of the frontier closed-source labs offer (and without the fear that your job might be met with a refusal at some random point where you're deep in a 2 hour job) -re: the above point...for this reason, a lot of US companies (inferact included!) choose to use open-weight models over their closed-source alternatives. this also isn't limited to internal workloads / research - on a recent a16z podcast the team at Decagon spoke about how something like 90% of their customer service ai agents run on open-weight models that they've fine-tuned. -we should really appreciate how many companies/teams came out researchers fascinated by the wave of very small open-weight models that were being distilled from e.g. gpt-3.5 and earlier models in 2022/2023 (prior to the release of chatGPT!). these small models motivated the development of pagedattention, which then led to vlmm/inferact (at other layers of the stack with similar origin stories, you can look at teams like openrouter or ollama). in other words, we have open-weight models to thank for a bunch of the orchestration infra we now rely on. i think yet another, indirect, way we can point to open-source/weight infra pushing the frontier forward. anyway, a lot more in this convo, it was a lot of fun!

learned a lot from this conversation with Simon Mo and Matt Bornstein. biggest takeaways for me: -there are a lot of reasons why we should like open-weight models. a lot of these arguments stop at handwavy things like "what if the labs stop releasing frontier models to the public" or "it's lower cost." but simon's position as lead maintainer of vLLM and CEO of Inferact give him authority to talk about some of the other, more interesting and concrete reasons to pay attention to open-weight models, namely that they allow end-users to calibrate latency / other performance metrics with way more customizability than what any of the frontier closed-source labs offer (and without the fear that your job might be met with a refusal at some random point where you're deep in a 2 hour job) -re: the above point...for this reason, a lot of US companies (inferact included!) choose to use open-weight models over their closed-source alternatives. this also isn't limited to internal workloads / research - on a recent a16z podcast the team at Decagon spoke about how something like 90% of their customer service ai agents run on open-weight models that they've fine-tuned. -we should really appreciate how many companies/teams came out researchers fascinated by the wave of very small open-weight models that were being distilled from e.g. gpt-3.5 and earlier models in 2022/2023 (prior to the release of chatGPT!). these small models motivated the development of pagedattention, which then led to vlmm/inferact (at other layers of the stack with similar origin stories, you can look at teams like openrouter or ollama). in other words, we have open-weight models to thank for a bunch of the orchestration infra we now rely on. i think yet another, indirect, way we can point to open-source/weight infra pushing the frontier forward. anyway, a lot more in this convo, it was a lot of fun!

Elena

12,917 views • 1 month ago

Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.

Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.

NVIDIA AI

1,329,222 views • 3 months ago

$WDC CEO says storage demand and requirement compound and grow whether AI runs on US frontier models or cheaper open-source ones. "Whether it's the traditional frontier models that you've seen in the U.S. that are premium-based, or some of the new open-source, more economical models, the underlying requirement is still that they're going to require a lot of data to support the training of these models." "These models are going to generate a lot more data that requires storage for us." "So we view it as very positive, because even as compute maybe drives greater efficiency in terms of compute and memory resources, the requirement for storage is just going to compound and grow." "So we view it very positively for demand going forward."

$WDC CEO says storage demand and requirement compound and grow whether AI runs on US frontier models or cheaper open-source ones. "Whether it's the traditional frontier models that you've seen in the U.S. that are premium-based, or some of the new open-source, more economical models, the underlying requirement is still that they're going to require a lot of data to support the training of these models." "These models are going to generate a lot more data that requires storage for us." "So we view it as very positive, because even as compute maybe drives greater efficiency in terms of compute and memory resources, the requirement for storage is just going to compound and grow." "So we view it very positively for demand going forward."

Fireside Alpha

14,114 views • 1 month ago

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Base Labs

227,367 views • 9 days ago

Aligned with Matan Grinberg's intuition that in the next 12 months, 90% of tokens could be going to open models. We're seeing more and more companies using frontier APIs for experimenting and open-source or private models for production and larger workloads!

Aligned with Matan Grinberg's intuition that in the next 12 months, 90% of tokens could be going to open models. We're seeing more and more companies using frontier APIs for experimenting and open-source or private models for production and larger workloads!

clem 🤗

24,138 views • 2 months ago

Chamath is making one of the most important business arguments of 2026. Half of large US companies right now cannot generate returns that exceed their cost of capital, which has normalized back to its long run average of 8 to 11%. Another one in seven companies globally is stuck generating persistent returns between 1 and 5% and most businesses don't have room for error and in this environment walks every frontier AI lab saying the same thing, give us your data, your workflows, your processes and our model will make everything better. And companies by the millions said yes. What they didn't fully account for is what happens on the other side of that door. Every time an employee runs a query through a frontier model API, the prompt goes through external servers, workflows, customer data, pricing logic, internal processes, all of it transmitted through a third party. As Alex Karp said companies are spending on tokens while handing over the exact proprietary advantages that make their business worth owning. Microsoft blocked internal use of Anthropic's Claude Fable 5 but over its 30-day data retention policy and the largest software company in the world decided a frontier model's data handling was too risky for its own employees. A US government action revoked access to another frontier model for foreign nationals overnight. Now here's where the cost math becomes impossible to ignore. Deutsche Bank calculated a roughly 65x cost gap between frontier models like Claude Fable 5 at ~$3.25 per task and open-source alternatives at ~$0.05. For 90% of everyday enterprise tasks, performance is comparable. Open-weight models now match closed frontier systems on core agent tasks at roughly one-tenth the cost, a high-volume deployment that costs $250/day on Claude runs at $12/day on an open-source equivalent. Chamath Palihapitiya tested this directly by running a standard enterprise code migration task through an orchestration layer wrapping an open-source model came in 16.4x cheaper than using a frontier model directly.

Chamath is making one of the most important business arguments of 2026. Half of large US companies right now cannot generate returns that exceed their cost of capital, which has normalized back to its long run average of 8 to 11%. Another one in seven companies globally is stuck generating persistent returns between 1 and 5% and most businesses don't have room for error and in this environment walks every frontier AI lab saying the same thing, give us your data, your workflows, your processes and our model will make everything better. And companies by the millions said yes. What they didn't fully account for is what happens on the other side of that door. Every time an employee runs a query through a frontier model API, the prompt goes through external servers, workflows, customer data, pricing logic, internal processes, all of it transmitted through a third party. As Alex Karp said companies are spending on tokens while handing over the exact proprietary advantages that make their business worth owning. Microsoft blocked internal use of Anthropic's Claude Fable 5 but over its 30-day data retention policy and the largest software company in the world decided a frontier model's data handling was too risky for its own employees. A US government action revoked access to another frontier model for foreign nationals overnight. Now here's where the cost math becomes impossible to ignore. Deutsche Bank calculated a roughly 65x cost gap between frontier models like Claude Fable 5 at ~$3.25 per task and open-source alternatives at ~$0.05. For 90% of everyday enterprise tasks, performance is comparable. Open-weight models now match closed frontier systems on core agent tasks at roughly one-tenth the cost, a high-volume deployment that costs $250/day on Claude runs at $12/day on an open-source equivalent. Chamath Palihapitiya tested this directly by running a standard enterprise code migration task through an orchestration layer wrapping an open-source model came in 16.4x cheaper than using a frontier model directly.

Milk Road AI

281,771 views • 2 months ago

.Oliver Cameron says one of the biggest applications for world models will be training AI inside simulated environments. "One of the killer applications of world models will be as a learning environment where language models and other types of AI can learn. It gets very much philosophical."

.Oliver Cameron says one of the biggest applications for world models will be training AI inside simulated environments. "One of the killer applications of world models will be as a learning environment where language models and other types of AI can learn. It gets very much philosophical."

TBPN

40,172 views • 1 month ago

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

Harry Stebbings

63,738 views • 2 months ago

Quick chat with dylan ツ (Dylan Bristot, GTM @ $NBIS). Also on YouTube (link in first comment) for those who prefer to watch/listen there. Timestamps 00:00 – Dylan's role at Nebius and Nebius Token Factory 01:48 – Dylan's investing philosophy and portfolio approach 05:07 – How working in AI infrastructure influences his investing 08:29 – Training vs. inference and why inference demand could explode 13:22 – Enterprise AI adoption: from POCs to production 18:02 – Open-source vs. closed/frontier models 24:44 – The economics of open vs. closed AI models 29:27 – Where the next AI infrastructure bottlenecks could emerge 31:22 – Dylan's AI Bottlenecks project and approach to stock selection 34:06 – Closing thoughts Key Insights (AI Summary, so you don't have to copy paste and prompt for exactly that ;D) “I seem to like areas where the demand really looks kind of secular, but the supply is genuinely hard to create.” → Implication: The most attractive AI trades may sit in physical bottlenecks where supply cannot quickly respond to demand. “The bottleneck is who has the pricing power and kind of what might get commoditized and where the concentrate might move next.” → Implication: Value capture across the AI stack will keep shifting as individual layers become scarce or commoditized. “Training creates the intelligence and then the inference actually monetizes and distributes.” → Implication: Training and inference are complementary, rather than one ultimately replacing the other. “One user action can become dozens or hundreds of model calls, tools calls, and like verification steps, retries.” → Implication: Agentic AI can drive token consumption far faster than user growth alone would suggest. “The best infra for making any model and the best infra for serving a billion interactions are not necessarily the same.” → Implication: Training and inference could increasingly require different hardware and infrastructure architectures. “The Frontier Labs might be incentivized to run more and more of the inference of these models for internal research instead of providing it to external people.” → Implication: The most capable models and their compute could increasingly be used internally to accelerate frontier research rather than monetized externally. “Enterprise AI adoption is actually much further along than a lot of people kind of think. But probably less mature than the headlines suggest.” → Implication: Enterprise demand is real, but deployment maturity still has significant room to improve. “The POC problem might be solved for a lot of companies, but the production problem isn’t yet.” → Implication: The enterprise bottleneck is shifting from proving AI works to deploying it reliably, securely and economically at scale. “They feel like it’s time for them to actually not only integrate AI, but build some sort of moat out of the AI.” → Implication: Enterprises increasingly want proprietary AI systems built around their own data rather than simply consuming generic models. “The more autonomous the software becomes, the more infra discipline you need underneath it.” → Implication: Agents increase the importance of inference cost, reliability and infrastructure optimization. “Maybe I have fifteen different versions of very different LLMs, fine tuned on fifteen different kinds of tasks that I’m operating across my business, instead of having a one model fits all.” → Implication: Enterprise AI could evolve toward many specialized models rather than one frontier model handling every workload. “I don’t necessarily think it’s open versus closed. That might be the wrong framing.” → Implication: Open and closed models can coexist because they optimize for different customer needs. “Historically the problem was that that control came with a massive operational tax.” → Implication: Better inference infrastructure can make open models materially more competitive by removing the complexity traditionally associated with running them. “I don’t think open needs to beat the best closed model on every single benchmark. It just basically needs to be good enough for the workload of the given customer while offering a much better combination of control, cost, and deployment flexibility.” → Implication: For production AI, workload-specific economics may matter more than having the absolute smartest model. “Maybe actually the bulk of tokens generated in the future might come from open models.” → Implication: Frontier intelligence could remain dominated by closed labs even while open models capture most production inference volume. “I could really imagine frontier intelligence being really concentrated while most of the production inference becomes super fragmented.” → Implication: AI could consolidate at the intelligence layer while fragmenting heavily at the inference layer across models, GPUs, providers and regions. “I don’t think that necessarily means the margins of open source will be much worse than the ones of closed source.” → Implication: Optimization can potentially make open-model inference highly profitable despite lower pricing. “I think now we’re probably in the middle of phase two... everything feeding the accelerator.” → Implication: The AI trade is broadening beyond GPUs toward networking, packaging, data centers, electrical equipment and power. “It’s no longer about the megawatts, about energized megawatts.” → Implication: Available power on paper matters less than how quickly that power can actually be delivered to operating AI infrastructure. “It’s increasingly about utilisation and conversion now and like how efficiently you convert expensive infra into actual useful AI work.” → Implication: Infrastructure efficiency and utilization become increasingly important as the absolute amount of deployed AI infrastructure grows. “The market tends to really notice demand before it notices what demand breaks.” → Implication: Second-order bottlenecks may offer some of the most interesting opportunities in the next phase of the AI buildout. “The interesting question now is which part of the mine breaks next?” → Implication: Finding the next constraint in the AI supply chain may matter more than simply identifying continued AI demand.

Quick chat with dylan ツ (Dylan Bristot, GTM @ $NBIS). Also on YouTube (link in first comment) for those who prefer to watch/listen there. Timestamps 00:00 – Dylan's role at Nebius and Nebius Token Factory 01:48 – Dylan's investing philosophy and portfolio approach 05:07 – How working in AI infrastructure influences his investing 08:29 – Training vs. inference and why inference demand could explode 13:22 – Enterprise AI adoption: from POCs to production 18:02 – Open-source vs. closed/frontier models 24:44 – The economics of open vs. closed AI models 29:27 – Where the next AI infrastructure bottlenecks could emerge 31:22 – Dylan's AI Bottlenecks project and approach to stock selection 34:06 – Closing thoughts Key Insights (AI Summary, so you don't have to copy paste and prompt for exactly that ;D) “I seem to like areas where the demand really looks kind of secular, but the supply is genuinely hard to create.” → Implication: The most attractive AI trades may sit in physical bottlenecks where supply cannot quickly respond to demand. “The bottleneck is who has the pricing power and kind of what might get commoditized and where the concentrate might move next.” → Implication: Value capture across the AI stack will keep shifting as individual layers become scarce or commoditized. “Training creates the intelligence and then the inference actually monetizes and distributes.” → Implication: Training and inference are complementary, rather than one ultimately replacing the other. “One user action can become dozens or hundreds of model calls, tools calls, and like verification steps, retries.” → Implication: Agentic AI can drive token consumption far faster than user growth alone would suggest. “The best infra for making any model and the best infra for serving a billion interactions are not necessarily the same.” → Implication: Training and inference could increasingly require different hardware and infrastructure architectures. “The Frontier Labs might be incentivized to run more and more of the inference of these models for internal research instead of providing it to external people.” → Implication: The most capable models and their compute could increasingly be used internally to accelerate frontier research rather than monetized externally. “Enterprise AI adoption is actually much further along than a lot of people kind of think. But probably less mature than the headlines suggest.” → Implication: Enterprise demand is real, but deployment maturity still has significant room to improve. “The POC problem might be solved for a lot of companies, but the production problem isn’t yet.” → Implication: The enterprise bottleneck is shifting from proving AI works to deploying it reliably, securely and economically at scale. “They feel like it’s time for them to actually not only integrate AI, but build some sort of moat out of the AI.” → Implication: Enterprises increasingly want proprietary AI systems built around their own data rather than simply consuming generic models. “The more autonomous the software becomes, the more infra discipline you need underneath it.” → Implication: Agents increase the importance of inference cost, reliability and infrastructure optimization. “Maybe I have fifteen different versions of very different LLMs, fine tuned on fifteen different kinds of tasks that I’m operating across my business, instead of having a one model fits all.” → Implication: Enterprise AI could evolve toward many specialized models rather than one frontier model handling every workload. “I don’t necessarily think it’s open versus closed. That might be the wrong framing.” → Implication: Open and closed models can coexist because they optimize for different customer needs. “Historically the problem was that that control came with a massive operational tax.” → Implication: Better inference infrastructure can make open models materially more competitive by removing the complexity traditionally associated with running them. “I don’t think open needs to beat the best closed model on every single benchmark. It just basically needs to be good enough for the workload of the given customer while offering a much better combination of control, cost, and deployment flexibility.” → Implication: For production AI, workload-specific economics may matter more than having the absolute smartest model. “Maybe actually the bulk of tokens generated in the future might come from open models.” → Implication: Frontier intelligence could remain dominated by closed labs even while open models capture most production inference volume. “I could really imagine frontier intelligence being really concentrated while most of the production inference becomes super fragmented.” → Implication: AI could consolidate at the intelligence layer while fragmenting heavily at the inference layer across models, GPUs, providers and regions. “I don’t think that necessarily means the margins of open source will be much worse than the ones of closed source.” → Implication: Optimization can potentially make open-model inference highly profitable despite lower pricing. “I think now we’re probably in the middle of phase two... everything feeding the accelerator.” → Implication: The AI trade is broadening beyond GPUs toward networking, packaging, data centers, electrical equipment and power. “It’s no longer about the megawatts, about energized megawatts.” → Implication: Available power on paper matters less than how quickly that power can actually be delivered to operating AI infrastructure. “It’s increasingly about utilisation and conversion now and like how efficiently you convert expensive infra into actual useful AI work.” → Implication: Infrastructure efficiency and utilization become increasingly important as the absolute amount of deployed AI infrastructure grows. “The market tends to really notice demand before it notices what demand breaks.” → Implication: Second-order bottlenecks may offer some of the most interesting opportunities in the next phase of the AI buildout. “The interesting question now is which part of the mine breaks next?” → Implication: Finding the next constraint in the AI supply chain may matter more than simply identifying continued AI demand.

Daniel Koss

49,970 views • 11 days ago

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 views • 5 months ago

$The future of AI deployment should be fast, affordable, and owned by all. Introducing Gradient Cloud, the all-in-one station for building with AI, powered by the Gradient distributed AI stack. Run frontier models at production speed for a fraction of the cost.$

The future of AI deployment should be fast, affordable, and owned by all. Introducing Gradient Cloud, the all-in-one station for building with AI, powered by the Gradient distributed AI stack. Run frontier models at production speed for a fraction of the cost.

Gradient

128,388 views • 1 year ago

Interesting that in three of my recent conversations with Krishna at Anthropic, Dylan Patel, and Gavin Baker, each said that frontier tokens are capturing the majority of the economic value. Gavin: "An overwhelming amount of the economic returns to AI at the model layer have been at the frontier. That's surprising to me, and I think it's been surprising to a lot of people. This is one of the most important questions to be answered, and you need to have a hypothesis on it as an investor." Krishna: "We think the returns to frontier intelligence are extremely high. Customers invest really heavily in more tokens with the newer models. The ones at the frontier clearly are capturing this economic value, driving meaningful ROI for customers. The returns to frontier intelligence are not slowing down." Dylan: "No one gives a crap about GPT-4 class models. They want the frontier because the frontier lets them create the economically valuable things."

Interesting that in three of my recent conversations with Krishna at Anthropic, Dylan Patel, and Gavin Baker, each said that frontier tokens are capturing the majority of the economic value. Gavin: "An overwhelming amount of the economic returns to AI at the model layer have been at the frontier. That's surprising to me, and I think it's been surprising to a lot of people. This is one of the most important questions to be answered, and you need to have a hypothesis on it as an investor." Krishna: "We think the returns to frontier intelligence are extremely high. Customers invest really heavily in more tokens with the newer models. The ones at the frontier clearly are capturing this economic value, driving meaningful ROI for customers. The returns to frontier intelligence are not slowing down." Dylan: "No one gives a crap about GPT-4 class models. They want the frontier because the frontier lets them create the economically valuable things."

Patrick OShaughnessy

121,392 views • 3 months ago

$QCOM buying Modular was about more than adding another AI asset. I sat down with Modular CEO Chris Lattner and Qualcomm CEO Cristiano R. Amon on why the industry needs a better software model for heterogeneous compute and how Qualcomm + Modular intend to attack it. If they can make it dramatically easier for developers to target CPUs, GPUs and accelerators without being locked into a single architecture, this could matter a lot.

$QCOM buying Modular was about more than adding another AI asset. I sat down with Modular CEO Chris Lattner and Qualcomm CEO Cristiano R. Amon on why the industry needs a better software model for heterogeneous compute and how Qualcomm + Modular intend to attack it. If they can make it dramatically easier for developers to target CPUs, GPUs and accelerators without being locked into a single architecture, this could matter a lot.

Patrick Moorhead

55,068 views • 19 days ago