Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

SITUATION EXPLAINED: Model fusion, or combining several frontier models into one system, may be better than any single model, sometimes at a lower price. ⿻ Andrew Trask, senior researcher at Google DeepMind: "Model fusion is basically this idea that, when you combine models, they can give you higher accuracy... than any of the individual models that you're using." "Some recent studies are actually showing they're able to deliver such a significant jump in accuracy that it's actually a lower cost option than buying some of the premium models." "So for example, there's a few different companies, showing the higher accuracy claims. So Sakana, OpenRouter are all releasing results in the last year/couple months, and we replicated some of it internally at OpenMined." "Some of the most surprising stuff, though, is that they're reaching past Fable level quality with non-Fable models, and in the case of OpenRouter, at half the price."show more

MTS

386,440 subscribers

20,167 views • 1 month ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

We just beat the best frontier models at a quarter of the price. Now you can too, in one API call. Today we're launching Merge Fusion. Send one prompt to a panel of models, and a judge writes one answer better than any of them alone. Fusion beats Fable 5 outright at 1/4th the cost. It also sets the highest score on the DRACO benchmark.

We just beat the best frontier models at a quarter of the price. Now you can too, in one API call. Today we're launching Merge Fusion. Send one prompt to a panel of models, and a judge writes one answer better than any of them alone. Fusion beats Fable 5 outright at 1/4th the cost. It also sets the highest score on the DRACO benchmark.

Shensi Ding

398,319 views • 19 days ago

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 views • 2 years ago

🇺🇸 DAVID SACKS: MODELS ARE LEAPFROGGING EACH OTHER AND BECOMING SPECIALIZED "The model companies are still leapfrogging each other to some degree with their latest versions. And again, that should not be possible if one model achieved rapid takeoff. The other thing you're seeing is that it's not like one model is dominating all the other models. You're seeing that models are developing areas of competitive advantage. They're becoming increasingly specialized." Source: The All-In Podcast David Sacks

🇺🇸 DAVID SACKS: MODELS ARE LEAPFROGGING EACH OTHER AND BECOMING SPECIALIZED "The model companies are still leapfrogging each other to some degree with their latest versions. And again, that should not be possible if one model achieved rapid takeoff. The other thing you're seeing is that it's not like one model is dominating all the other models. You're seeing that models are developing areas of competitive advantage. They're becoming increasingly specialized." Source: The All-In Podcast David Sacks

Mario Nawfal

45,461 views • 11 months ago

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnan

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnan

MTS

71,286 views • 22 days ago

.Marc Andreessen 🇺🇸 on why God models don't eat all AI usage: "There's one version of the world in which you have a small number of what we sometimes call God models... millions of processors, with billions or hundreds of billions or trillions of dollars of CapEx and training, investment behind it." "That's the equivalent of the world of search that we've been in for the last basically 20 years." "There's another scenario, what they sometimes call a race to the bottom, which is no, actually what happens is intelligence commoditizes." "The analogy there would be maybe what's happened in the microchip market... A really good one costs a few thousand dollars, there's companies like Arm that sell them for pennies. And you can put a microchip in anything, and it's super cheap, and everybody does it." "My guess basically is what we're about to see is something a lot like the microchip or the internet." "You'll be able to basically summon intelligence at some level into any application you need for basically very cheap or ultimately basically the same as free. And then you'll have the God models in the background when you need the super genius, but for most things, you don't actually need a super genius."

.Marc Andreessen 🇺🇸 on why God models don't eat all AI usage: "There's one version of the world in which you have a small number of what we sometimes call God models... millions of processors, with billions or hundreds of billions or trillions of dollars of CapEx and training, investment behind it." "That's the equivalent of the world of search that we've been in for the last basically 20 years." "There's another scenario, what they sometimes call a race to the bottom, which is no, actually what happens is intelligence commoditizes." "The analogy there would be maybe what's happened in the microchip market... A really good one costs a few thousand dollars, there's companies like Arm that sell them for pennies. And you can put a microchip in anything, and it's super cheap, and everybody does it." "My guess basically is what we're about to see is something a lot like the microchip or the internet." "You'll be able to basically summon intelligence at some level into any application you need for basically very cheap or ultimately basically the same as free. And then you'll have the God models in the background when you need the super genius, but for most things, you don't actually need a super genius."

a16z

104,451 views • 2 months ago

SITUATION EXPLAINED: 70% of frontier model queries could run locally for free. clem 🤗, co-founder and CEO of Hugging Face: "There was an interesting study from Stanford published last year showing that 70% of the queries that people ask to ChatGPT could be accurately answered locally on your laptop. For free." "Most of the AI workloads that people do today with frontier models could be done by models that are cheaper, faster, more customizable, more controllable. And they don't do it because frankly it's a pain to pick the right model." "You're subsidized so you don't have to care because you have your subscription, so you route everything to Einstein. 'Hey Einstein, what's the weather today?'" "In normal life, he would be like, 'I'm not answering your silly question.' But because it's AI and subsidized by the AI labs, all the questions are getting routed to Einstein versus in an ideal world you have different models that are more specialized and better at answering your questions in different domains." "It's possible that routing is gonna redistribute a lot of the value capture from frontier models to a more long tail of models. It's like AI maturing... we are in the first phase where it's very simple, everyone using just one giant model. Now we're moving to the second phase."

SITUATION EXPLAINED: 70% of frontier model queries could run locally for free. clem 🤗, co-founder and CEO of Hugging Face: "There was an interesting study from Stanford published last year showing that 70% of the queries that people ask to ChatGPT could be accurately answered locally on your laptop. For free." "Most of the AI workloads that people do today with frontier models could be done by models that are cheaper, faster, more customizable, more controllable. And they don't do it because frankly it's a pain to pick the right model." "You're subsidized so you don't have to care because you have your subscription, so you route everything to Einstein. 'Hey Einstein, what's the weather today?'" "In normal life, he would be like, 'I'm not answering your silly question.' But because it's AI and subsidized by the AI labs, all the questions are getting routed to Einstein versus in an ideal world you have different models that are more specialized and better at answering your questions in different domains." "It's possible that routing is gonna redistribute a lot of the value capture from frontier models to a more long tail of models. It's like AI maturing... we are in the first phase where it's very simple, everyone using just one giant model. Now we're moving to the second phase."

MTS

49,884 views • 1 month ago

China is once again unsettling its American competitors in the AI race. Chinese labs have returned to the spotlight with the unveiling of GLM 5.2, described as the most powerful AI model ever trained in China — and one of the leading open‑source models worldwide. 🔍 Key highlights of “GLM 5.2”: ♦️ It is considered the strongest AI model developed in China to date, and ranks among the smartest open‑source models available. ♦️ It operates at a cost less than one‑tenth that of Anthropic’s latest models, making it a far more economical option. ♦️ It is likely to attract developers and companies seeking powerful, low‑cost models — especially amid growing technical and geopolitical restrictions. ♦️ Some evaluations place the model among the top globally, trailing OpenAI’s models while outperforming several competitors in specialized benchmarks and performance tests.

China is once again unsettling its American competitors in the AI race. Chinese labs have returned to the spotlight with the unveiling of GLM 5.2, described as the most powerful AI model ever trained in China — and one of the leading open‑source models worldwide. 🔍 Key highlights of “GLM 5.2”: ♦️ It is considered the strongest AI model developed in China to date, and ranks among the smartest open‑source models available. ♦️ It operates at a cost less than one‑tenth that of Anthropic’s latest models, making it a far more economical option. ♦️ It is likely to attract developers and companies seeking powerful, low‑cost models — especially amid growing technical and geopolitical restrictions. ♦️ Some evaluations place the model among the top globally, trailing OpenAI’s models while outperforming several competitors in specialized benchmarks and performance tests.

China pulse 🇨🇳

14,293 views • 1 month ago

The Model YL was on display in Naperville, Illinois last weekend. 😍 Illinois was one of the first two locations to display this car to the public… well, at least here in the USA. 😉 This is a short review of the car. I cover some of the features that are different from the Model Y Standard and Premium models. ✅

The Model YL was on display in Naperville, Illinois last weekend. 😍 Illinois was one of the first two locations to display this car to the public… well, at least here in the USA. 😉 This is a short review of the car. I cover some of the features that are different from the Model Y Standard and Premium models. ✅

TechGeek Tesla 🔋⚡️

35,707 views • 15 days ago

There's an economics theorem called Alchian-Allen. And it has the very interesting implication that AI labs will be able to charge *higher* margins on their best models as compute gets scarcer. As compute gets more expensive, the cost of running any model goes up. So you might as well pay a bit more to make sure you're running the very best model. Which means the economics of being at the frontier improve, because if you’re not running the very best model, then you’re underutilizing this very precious compute. This pushes the AI model market towards winner-take-all; if you're the best, you can get away with charging an even higher margin. Dylan Patel tells me that we’re already seeing this today: all the revenue in the industry is on the best models. That’s the Alchian-Allen effect. If there’s a cost increase that’s roughly the same for all products, then the relative difference in price between higher and lower quality goods actually goes down. Consumers become relatively more willing to pay for the premium product. And it means that as the compute shortage hits, AI labs can capture more margin - not less, as you might expect - because consumers are choosing premium models more often.

There's an economics theorem called Alchian-Allen. And it has the very interesting implication that AI labs will be able to charge higher margins on their best models as compute gets scarcer. As compute gets more expensive, the cost of running any model goes up. So you might as well pay a bit more to make sure you're running the very best model. Which means the economics of being at the frontier improve, because if you’re not running the very best model, then you’re underutilizing this very precious compute. This pushes the AI model market towards winner-take-all; if you're the best, you can get away with charging an even higher margin. Dylan Patel tells me that we’re already seeing this today: all the revenue in the industry is on the best models. That’s the Alchian-Allen effect. If there’s a cost increase that’s roughly the same for all products, then the relative difference in price between higher and lower quality goods actually goes down. Consumers become relatively more willing to pay for the premium product. And it means that as the compute shortage hits, AI labs can capture more margin - not less, as you might expect - because consumers are choosing premium models more often.

Dwarkesh Patel

323,500 views • 5 months ago

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

MTS

61,234 views • 22 days ago

David Sacks: Anthropic Wants to Ban Open Source AI in America Listen to All-In to stay a few weeks (or months) ahead of everyone else David Sacks back in May: “I think where it's all leading to is an effort to ban open source models. There's a lot of breadcrumbs leading here. You look at a lot of the rhetoric around how models need to have guardrails, and that with open source models, the guardrails can be removed and therefore they're dangerous. You see this rhetoric already in Anthropic's blog posts. Any threat that they describe, they kind of go out of their way to take that shot at open source models. They're trying to create ideas or put predicate facts in the public record to justify an action later on. I think it's just a matter of time before they feel like they're at a position where maybe they can push for that type of ban directly.”

David Sacks: Anthropic Wants to Ban Open Source AI in America Listen to All-In to stay a few weeks (or months) ahead of everyone else David Sacks back in May: “I think where it's all leading to is an effort to ban open source models. There's a lot of breadcrumbs leading here. You look at a lot of the rhetoric around how models need to have guardrails, and that with open source models, the guardrails can be removed and therefore they're dangerous. You see this rhetoric already in Anthropic's blog posts. Any threat that they describe, they kind of go out of their way to take that shot at open source models. They're trying to create ideas or put predicate facts in the public record to justify an action later on. I think it's just a matter of time before they feel like they're at a position where maybe they can push for that type of ban directly.”

The All-In Podcast

86,749 views • 20 days ago

Sakana AI CEO David Ha hardmaru: "If you assemble a team of Avengers, that team will be better than the individual heroes. The same with AI." Sakana Fugu, their new AI “model” goes toe-to-toe with Mythos and Fable 5 on a few benchmarks — without training a single frontier model of its own. Fugu orchestrates between different frontier AI models to achieve higher results. On the real risk, David adds: "I'm more concerned about the structure of dependency… you don't want to be tied to a single particular AI model or a single particular vendor." An exclusive 1:1 with CROSS DIG’s 中川雅博 / Masahiro Nakagawa @ TBS CROSS DIG #SakanaAI #SakanaFugu

Sakana AI CEO David Ha hardmaru: "If you assemble a team of Avengers, that team will be better than the individual heroes. The same with AI." Sakana Fugu, their new AI “model” goes toe-to-toe with Mythos and Fable 5 on a few benchmarks — without training a single frontier model of its own. Fugu orchestrates between different frontier AI models to achieve higher results. On the real risk, David adds: "I'm more concerned about the structure of dependency… you don't want to be tied to a single particular AI model or a single particular vendor." An exclusive 1:1 with CROSS DIG’s 中川雅博 / Masahiro Nakagawa @ TBS CROSS DIG #SakanaAI #SakanaFugu

TBS CROSS DIG with Bloomberg

78,986 views • 1 month ago

OpenAI's Chief Futurist Joshua Achiam breaks with the consensus on takeoff, arguing intelligence hits a physical ceiling and then raw compute decides who wins: "People expect that there's no ceiling for the amount of intelligence that you could have in a model, and they think of recursive self-improvement as this loop that's gonna happen at some point or another. RSI starts happening and model intelligence takes off and it goes to the moon, and they don't see a ceiling." "I think on physical grounds, there's gotta be a maximum amount of computation that you can have per unit volume and energy in the physical universe. And that sort of implies there's a maximum amount of intelligence per unit volume and unit of energy." "If that's the case, eventually, seeing how fast AI model capabilities are increasing right now, eventually everyone hits that saturation point and everyone's got roughly equivalently capable models from a raw intelligence perspective." "The open source frontier lags the closed source frontier by some number of months, but the fact that it's months is crazy. So eventually everyone is probably working with equally maximally capable models, and then it's amount of compute that you're able to throw at a problem that determines who wins." Joshua Achiam

OpenAI's Chief Futurist Joshua Achiam breaks with the consensus on takeoff, arguing intelligence hits a physical ceiling and then raw compute decides who wins: "People expect that there's no ceiling for the amount of intelligence that you could have in a model, and they think of recursive self-improvement as this loop that's gonna happen at some point or another. RSI starts happening and model intelligence takes off and it goes to the moon, and they don't see a ceiling." "I think on physical grounds, there's gotta be a maximum amount of computation that you can have per unit volume and energy in the physical universe. And that sort of implies there's a maximum amount of intelligence per unit volume and unit of energy." "If that's the case, eventually, seeing how fast AI model capabilities are increasing right now, eventually everyone hits that saturation point and everyone's got roughly equivalently capable models from a raw intelligence perspective." "The open source frontier lags the closed source frontier by some number of months, but the fact that it's months is crazy. So eventually everyone is probably working with equally maximally capable models, and then it's amount of compute that you're able to throw at a problem that determines who wins." Joshua Achiam

MTS

60,213 views • 18 days ago

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

AI at Meta

54,428 views • 2 years ago

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

The All-In Podcast

79,718 views • 1 year ago

David Sacks says companies are trapped paying OpenAI & Anthropic because they can't figure out how to use open source models "I think enterprise CTOs would like to shift their token consumption to cheaper models for the obvious reason that it would be more efficient. They are seeing compute costs or token costs skyrocket right now, so everyone's trying to figure this out." "You also have the AI sovereignty issue that Alex Karp talked about. They're worried about giving up the secret sauce or the alpha in their business to a frontier lab that may one day be competing with them. "The problem is, I think in most cases, they don't have the technical ability to do it. Coinbase figured out how to do it. DoorDash figured out how to do it. They built a token routing system that allows them to send frontier tasks to frontier models and non frontier tasks to more mundane models. But I don't think your average enterprise has the technical capability to do that." "This is why the share of wallet of closed models, it actually increased. I think that open source went from 19% last year to 11% this year. So open source as a share of enterprise spending is actually decreasing." "I don't think that means usage is decreasing. I think usage is skyrocketing. It also may be the case that because the whole point of using an open model is you just pay for the compute costs, you don't have to pay a lab, so it may be that it's hard to measure that usage in terms of spend." "But nonetheless, anyone who's saying that these closed models are going to lose or are somehow losing, you're just not seeing it in the data."

David Sacks says companies are trapped paying OpenAI & Anthropic because they can't figure out how to use open source models "I think enterprise CTOs would like to shift their token consumption to cheaper models for the obvious reason that it would be more efficient. They are seeing compute costs or token costs skyrocket right now, so everyone's trying to figure this out." "You also have the AI sovereignty issue that Alex Karp talked about. They're worried about giving up the secret sauce or the alpha in their business to a frontier lab that may one day be competing with them. "The problem is, I think in most cases, they don't have the technical ability to do it. Coinbase figured out how to do it. DoorDash figured out how to do it. They built a token routing system that allows them to send frontier tasks to frontier models and non frontier tasks to more mundane models. But I don't think your average enterprise has the technical capability to do that." "This is why the share of wallet of closed models, it actually increased. I think that open source went from 19% last year to 11% this year. So open source as a share of enterprise spending is actually decreasing." "I don't think that means usage is decreasing. I think usage is skyrocketing. It also may be the case that because the whole point of using an open model is you just pay for the compute costs, you don't have to pay a lab, so it may be that it's hard to measure that usage in terms of spend." "But nonetheless, anyone who's saying that these closed models are going to lose or are somehow losing, you're just not seeing it in the data."

dnap

110,354 views • 1 month ago

Anthropic's Sholto Douglas describes the concept of a software-only singularity: "It's one where the models are far better at digital tasks than they are at physical ones. And so we see rapid change in the digital world with models, and relatively little change in the physical world. So information and software changes dramatically, and this ends up having some pretty weird effects." "It means that maybe like the drivers of what have been the last couple decades of progress in the economy turn around." "And I think we'll see that flow on into the physical world but at a delay. So you get much better at doing chip design. You get much better at training AI models. AI models get a lot faster. Chips get a lot better. The general economy gets a lot more efficient because the sort of information and message parsing that is much of the rest of the economy ends up becoming much more efficient." "But at the same time you don't yet have robots providing limitless physical abundance. Science probably progresses really fast up to the degree that you need interaction with labs or larger particle colliders or something like this. And then you go, 'Okay well I need to build the robots.'"

Anthropic's Sholto Douglas describes the concept of a software-only singularity: "It's one where the models are far better at digital tasks than they are at physical ones. And so we see rapid change in the digital world with models, and relatively little change in the physical world. So information and software changes dramatically, and this ends up having some pretty weird effects." "It means that maybe like the drivers of what have been the last couple decades of progress in the economy turn around." "And I think we'll see that flow on into the physical world but at a delay. So you get much better at doing chip design. You get much better at training AI models. AI models get a lot faster. Chips get a lot better. The general economy gets a lot more efficient because the sort of information and message parsing that is much of the rest of the economy ends up becoming much more efficient." "But at the same time you don't yet have robots providing limitless physical abundance. Science probably progresses really fast up to the degree that you need interaction with labs or larger particle colliders or something like this. And then you go, 'Okay well I need to build the robots.'"

TBPN

208,090 views • 6 months ago

DeepMind CEO Demis Hassabis says the AI bottleneck is already moving from model ideas to physical supply chains. "How much of the restriction at this point is physical? I am talking memory shortage, GPU shortage, and electricity." Demis: "That is constraining a lot of the deployment for sure. We see so much more demand for our models - Gemini and the other models that we have - than we can possibly serve." Then the less obvious part: "It does constrain a little bit the research, because you need a lot of chips to be able to experiment on new ideas at big enough scale that you can actually see if they are going to work." And on memory: "The whole supply chain is kind of strained... It still in the end actually comes down to a few suppliers of a few of the key components. Anywhere where there is some kind of constraint on the capacity, there is a sort of choke point." Important framing: The next AI race is not just who has the best model architecture. It is who can secure memory, chips, power, suppliers, and enough experimental capacity to find the next breakthrough before everyone else.

DeepMind CEO Demis Hassabis says the AI bottleneck is already moving from model ideas to physical supply chains. "How much of the restriction at this point is physical? I am talking memory shortage, GPU shortage, and electricity." Demis: "That is constraining a lot of the deployment for sure. We see so much more demand for our models - Gemini and the other models that we have - than we can possibly serve." Then the less obvious part: "It does constrain a little bit the research, because you need a lot of chips to be able to experiment on new ideas at big enough scale that you can actually see if they are going to work." And on memory: "The whole supply chain is kind of strained... It still in the end actually comes down to a few suppliers of a few of the key components. Anywhere where there is some kind of constraint on the capacity, there is a sort of choke point." Important framing: The next AI race is not just who has the best model architecture. It is who can secure memory, chips, power, suppliers, and enough experimental capacity to find the next breakthrough before everyone else.

Karl Mehta

25,716 views • 1 month ago

Demis Hassabis says world models are his longest standing passion and explains benefits vs. language models: ▫️ “I think language models are able to understand a lot about the world. More than we expected because language is actually probably richer than we thought. But there's still a lot about the spatial dynamics of the world, spatial awareness and the physical context we're in — and how that works mechanically — that is hard to describe in words and isn't generally described in corpuses of words. Alot of this is allied to learning from experience. There's a lot of things which you can't really describe something. You have to just experience it. Maybe the senses and so on are very hard to put into words. Whether that's motor angles and smell and these kinds of senses, it's very difficult to describe that in any kind of language.”▫️ This is what Demis and Google Deepmind is trying to solve with Genie. He also says that the video models (Veo) will play a part in training the world models and this is all key for AI robotics.

Demis Hassabis says world models are his longest standing passion and explains benefits vs. language models: ▫️ “I think language models are able to understand a lot about the world. More than we expected because language is actually probably richer than we thought. But there's still a lot about the spatial dynamics of the world, spatial awareness and the physical context we're in — and how that works mechanically — that is hard to describe in words and isn't generally described in corpuses of words. Alot of this is allied to learning from experience. There's a lot of things which you can't really describe something. You have to just experience it. Maybe the senses and so on are very hard to put into words. Whether that's motor angles and smell and these kinds of senses, it's very difficult to describe that in any kind of language.”▫️ This is what Demis and Google Deepmind is trying to solve with Genie. He also says that the video models (Veo) will play a part in training the world models and this is all key for AI robotics.

Bearly AI

123,703 views • 7 months ago

MIT PhD student Alex Zhang on why simple harness abstractions beat models trained around their own harness: "When these abstractions are implemented in a really simple way, basically what you see with Prime Agent is that you can slot in any model. We don't train any model around Prime Agent." "In a lot of cases, we see Prime Agent doing better than models trained around a certain harness. It's purely because these abstractions take advantage of the raw capabilities of the models." alex zhang Seth Karten Prime Intellect

MIT PhD student Alex Zhang on why simple harness abstractions beat models trained around their own harness: "When these abstractions are implemented in a really simple way, basically what you see with Prime Agent is that you can slot in any model. We don't train any model around Prime Agent." "In a lot of cases, we see Prime Agent doing better than models trained around a certain harness. It's purely because these abstractions take advantage of the raw capabilities of the models." alex zhang Seth Karten Prime Intellect

MTS

146,258 views • 5 days ago