Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

SITUATION EXPLAINED: Model fusion, or combining several frontier models into one system, may be better than any single model, sometimes at a lower price. ⿻ Andrew Trask, senior researcher at Google DeepMind: "Model fusion is basically this idea that, when you combine models, they can give you higher accuracy...

MTSlive's profile picture

MTS

503,513 subscribers

20,187 Aufrufe • vor 4 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von Niko
Nikovor 4 Monaten

@iamtrask @GoogleDeepMind

Profilbild von Maxx
Maxxvor 4 Monaten

@iamtrask @GoogleDeepMind This was obvious a long time ago.

Profilbild von Erika S
Erika Svor 4 Monaten

@iamtrask @GoogleDeepMind Basically mixture of experts at inference time. I'm curious how this compares to fine-tuning for domain-specific tasks though.

Ähnliche Videos

We just beat the best frontier models at a quarter of the price. Now you can too, in one API call. Today we're launching Merge Fusion. Send one prompt to a panel of models, and a judge writes one answer better than any of them alone. Fusion beats Fable 5 outright at 1/4th the cost. It also sets the highest score on the DRACO benchmark.
0:45

We just beat the best frontier models at a quarter of the price. Now you can too, in one API call. Today we're launching Merge Fusion. Send one prompt to a panel of models, and a judge writes one answer better than any of them alone. Fusion beats Fable 5 outright at 1/4th the cost. It also sets the highest score on the DRACO benchmark.

Shensi Ding

415,218 Aufrufe • vor 3 Monaten

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.
2:52

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 Aufrufe • vor 2 Jahren

JD Vance: "There are things at some of the AI companies you see happening, where you feel like there is some really, really weird spiritual dark energy. Like one of the companies that develops these models, when they get rid of one model and announce a new model, they actually hold a burial ceremony for the old model, and the employees apparently will cry."
1:24

JD Vance: "There are things at some of the AI companies you see happening, where you feel like there is some really, really weird spiritual dark energy. Like one of the companies that develops these models, when they get rid of one model and announce a new model, they actually hold a burial ceremony for the old model, and the employees apparently will cry."

Harlan Stewart

361,667 Aufrufe • vor 2 Monaten

OpenAI researcher David Holtz reveals workers who don't adapt their prompting to newer models leave 50% of the alpha on the table because half of AI's value comes from how you interact with it: "We ran a lab experiment. We give people the same task to do, but we give them models of different capabilities. How much of the improvement when we give people more capable models comes from the fact that we gave them a better model vs the fact that they actually update and adapt the way they prompt and interact with these models?" "For tasks where precision matters, where you have some output you're trying to get and you're trying to steer the model, which is different from a creative task like brainstorming." "For these steering-based tasks, when you give people a better model, about 50% of the improvement we see in how well they're able to complete the task comes from changes in how they interact with the model." "If you just kept the way you interact with the model static and just made the capabilities better and better, you'd leave 50% of the alpha basically on the table." OpenAI
2:59

OpenAI researcher David Holtz reveals workers who don't adapt their prompting to newer models leave 50% of the alpha on the table because half of AI's value comes from how you interact with it: "We ran a lab experiment. We give people the same task to do, but we give them models of different capabilities. How much of the improvement when we give people more capable models comes from the fact that we gave them a better model vs the fact that they actually update and adapt the way they prompt and interact with these models?" "For tasks where precision matters, where you have some output you're trying to get and you're trying to steer the model, which is different from a creative task like brainstorming." "For these steering-based tasks, when you give people a better model, about 50% of the improvement we see in how well they're able to complete the task comes from changes in how they interact with the model." "If you just kept the way you interact with the model static and just made the capabilities better and better, you'd leave 50% of the alpha basically on the table." OpenAI

MTS

15,590 Aufrufe • vor 3 Monaten

SemiAnalysis staffer Andrew Megalaa says materials science PhD collaborators who helped develop SA's GDPx benchmark was surprised by frontier models' capabilities, and even found they outperformed published state-of-the-art results on some tasks: "Our collaborators found that the models are much, much better than previously expected. And so they had a moment where they had to sit back and be like, 'This is actually better than a lot of the stuff people are putting out there in papers.'" "In fact, on one of the problems, Astra solved a subtask of a task better than a state-of-the-art published reference." "It's very clear that the models are very spiky in these categories. And are reaching above superhuman baselines."
1:25

SemiAnalysis staffer Andrew Megalaa says materials science PhD collaborators who helped develop SA's GDPx benchmark was surprised by frontier models' capabilities, and even found they outperformed published state-of-the-art results on some tasks: "Our collaborators found that the models are much, much better than previously expected. And so they had a moment where they had to sit back and be like, 'This is actually better than a lot of the stuff people are putting out there in papers.'" "In fact, on one of the problems, Astra solved a subtask of a task better than a state-of-the-art published reference." "It's very clear that the models are very spiky in these categories. And are reaching above superhuman baselines."

TBPN

14,921 Aufrufe • vor 11 Tagen

OpenAI researcher Lukasz Kaiser explains why he's optimistic AI can remain truly open: intelligence may ultimately work better as many distributed models than one giant model "If you look at the world, humans are the most amazing computers in the world. Our brains are still unmatched by the models, and we are not generalists, and there is not one big brain of humanity. It's a lot of separate people." "Given a fixed amount of data, the best way to learn from it is to have a lot of distributed models that are strong on its own, but even stronger when they're, If you look at some basic research, there are actually reasons to believe it, like ensembles and things like that." "Transformers are great when you feed them all of the internet, but we know there is an algorithm that's even better and it can learn from much smaller data and be amazing at the things it's learning. We are the proof. We have not found it yet, but we will." "All these big models, they're all the same, they're kind of boring. I think they'll be much more fun." OpenAI
2:52

OpenAI researcher Lukasz Kaiser explains why he's optimistic AI can remain truly open: intelligence may ultimately work better as many distributed models than one giant model "If you look at the world, humans are the most amazing computers in the world. Our brains are still unmatched by the models, and we are not generalists, and there is not one big brain of humanity. It's a lot of separate people." "Given a fixed amount of data, the best way to learn from it is to have a lot of distributed models that are strong on its own, but even stronger when they're, If you look at some basic research, there are actually reasons to believe it, like ensembles and things like that." "Transformers are great when you feed them all of the internet, but we know there is an algorithm that's even better and it can learn from much smaller data and be amazing at the things it's learning. We are the proof. We have not found it yet, but we will." "All these big models, they're all the same, they're kind of boring. I think they'll be much more fun." OpenAI

MTS

15,860 Aufrufe • vor 2 Monaten

.L3Harris says Palantir helped them beat frontier AI models in less than 2 days, at 95% lower cost: "When we fine-tuned open source models trained on our own data, we were able to outperform the frontier models in less than 48 hours." "The cost of our fine-tuned open source model was 95% lower than the frontier models we were using." "AI is a commodity. It's all about the data. We view our data as a corporate asset. It's our unique hard-earned knowledge." "We believe American defense companies should not be a vassal for frontier AI labs, handing over our data and institutional knowledge, hoping to rent back the intelligence it creates." " We own the model, we own the compute, we own the advantage."
1:19

.L3Harris says Palantir helped them beat frontier AI models in less than 2 days, at 95% lower cost: "When we fine-tuned open source models trained on our own data, we were able to outperform the frontier models in less than 48 hours." "The cost of our fine-tuned open source model was 95% lower than the frontier models we were using." "AI is a commodity. It's all about the data. We view our data as a corporate asset. It's our unique hard-earned knowledge." "We believe American defense companies should not be a vassal for frontier AI labs, handing over our data and institutional knowledge, hoping to rent back the intelligence it creates." " We own the model, we own the compute, we own the advantage."

Jawwwn

1,108,745 Aufrufe • vor 2 Monaten

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnan
3:48

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnan

MTS

71,286 Aufrufe • vor 3 Monaten

.Marc Andreessen 🇺🇸 on why God models don't eat all AI usage: "There's one version of the world in which you have a small number of what we sometimes call God models... millions of processors, with billions or hundreds of billions or trillions of dollars of CapEx and training, investment behind it." "That's the equivalent of the world of search that we've been in for the last basically 20 years." "There's another scenario, what they sometimes call a race to the bottom, which is no, actually what happens is intelligence commoditizes." "The analogy there would be maybe what's happened in the microchip market... A really good one costs a few thousand dollars, there's companies like Arm that sell them for pennies. And you can put a microchip in anything, and it's super cheap, and everybody does it." "My guess basically is what we're about to see is something a lot like the microchip or the internet." "You'll be able to basically summon intelligence at some level into any application you need for basically very cheap or ultimately basically the same as free. And then you'll have the God models in the background when you need the super genius, but for most things, you don't actually need a super genius."
1:33

.Marc Andreessen 🇺🇸 on why God models don't eat all AI usage: "There's one version of the world in which you have a small number of what we sometimes call God models... millions of processors, with billions or hundreds of billions or trillions of dollars of CapEx and training, investment behind it." "That's the equivalent of the world of search that we've been in for the last basically 20 years." "There's another scenario, what they sometimes call a race to the bottom, which is no, actually what happens is intelligence commoditizes." "The analogy there would be maybe what's happened in the microchip market... A really good one costs a few thousand dollars, there's companies like Arm that sell them for pennies. And you can put a microchip in anything, and it's super cheap, and everybody does it." "My guess basically is what we're about to see is something a lot like the microchip or the internet." "You'll be able to basically summon intelligence at some level into any application you need for basically very cheap or ultimately basically the same as free. And then you'll have the God models in the background when you need the super genius, but for most things, you don't actually need a super genius."

a16z

104,623 Aufrufe • vor 5 Monaten

a quant researcher at IMC (one of the oldest electronic market makers in the world) on a result that catches most candidates out in interviews. first, why "just predict further out, there's less noise" is wrong: "a number of people suggest they would try and predict one day into the future because there's less noise. i think they're thinking of some kind of central limit theorem idea." "but if you just think about what happens in markets ,the longer you look into the future, the more can happen. the more people can come and trade with you, the more idiosyncratic events." "looking further into the future has more variance and is harder to predict." then the counterintuitive part: "models trained for shorter term predictions can actually be better at predicting longer horizons than a model specifically trained on that target horizon." "you might train two models, one at 5 minutes, one at 10 minutes. and you might find the five minute model is actually better at predicting 10 minutes than the 10-minute model." why: "there's a battle between the signal you're trying to use getting diffused by noise the longer you look. it can be better to train on a shorter horizon which captures more of that short-term signal, and then leverage the conditional relationship between the 5 and 10-minute targets."
4:56

a quant researcher at IMC (one of the oldest electronic market makers in the world) on a result that catches most candidates out in interviews. first, why "just predict further out, there's less noise" is wrong: "a number of people suggest they would try and predict one day into the future because there's less noise. i think they're thinking of some kind of central limit theorem idea." "but if you just think about what happens in markets ,the longer you look into the future, the more can happen. the more people can come and trade with you, the more idiosyncratic events." "looking further into the future has more variance and is harder to predict." then the counterintuitive part: "models trained for shorter term predictions can actually be better at predicting longer horizons than a model specifically trained on that target horizon." "you might train two models, one at 5 minutes, one at 10 minutes. and you might find the five minute model is actually better at predicting 10 minutes than the 10-minute model." why: "there's a battle between the signal you're trying to use getting diffused by noise the longer you look. it can be better to train on a shorter horizon which captures more of that short-term signal, and then leverage the conditional relationship between the 5 and 10-minute targets."

Goshawk Trades

31,523 Aufrufe • vor 2 Monaten

.Decagon co-founder and CTO Ashwin Sreenivas says the choice between expensive frontier models and cheaper, less capable ones is a false trade-off: "Even if you have a 'dumber model,' you can get it to higher performance on that specific task." "When we fine-tune smaller, dumber models, it's that they're just not as general purpose, but on the specific task we want them to do, they actually outperform the large, smart, state-of-the-art models." "So we end up getting all three things. It is better at the task, it is cheaper, and it is faster." Ashwin Sreenivas
0:47

.Decagon co-founder and CTO Ashwin Sreenivas says the choice between expensive frontier models and cheaper, less capable ones is a false trade-off: "Even if you have a 'dumber model,' you can get it to higher performance on that specific task." "When we fine-tune smaller, dumber models, it's that they're just not as general purpose, but on the specific task we want them to do, they actually outperform the large, smart, state-of-the-art models." "So we end up getting all three things. It is better at the task, it is cheaper, and it is faster." Ashwin Sreenivas

a16z

36,122 Aufrufe • vor 3 Monaten

SITUATION EXPLAINED: 70% of frontier model queries could run locally for free. clem 🤗, co-founder and CEO of Hugging Face: "There was an interesting study from Stanford published last year showing that 70% of the queries that people ask to ChatGPT could be accurately answered locally on your laptop. For free." "Most of the AI workloads that people do today with frontier models could be done by models that are cheaper, faster, more customizable, more controllable. And they don't do it because frankly it's a pain to pick the right model." "You're subsidized so you don't have to care because you have your subscription, so you route everything to Einstein. 'Hey Einstein, what's the weather today?'" "In normal life, he would be like, 'I'm not answering your silly question.' But because it's AI and subsidized by the AI labs, all the questions are getting routed to Einstein versus in an ideal world you have different models that are more specialized and better at answering your questions in different domains." "It's possible that routing is gonna redistribute a lot of the value capture from frontier models to a more long tail of models. It's like AI maturing... we are in the first phase where it's very simple, everyone using just one giant model. Now we're moving to the second phase."
3:35

SITUATION EXPLAINED: 70% of frontier model queries could run locally for free. clem 🤗, co-founder and CEO of Hugging Face: "There was an interesting study from Stanford published last year showing that 70% of the queries that people ask to ChatGPT could be accurately answered locally on your laptop. For free." "Most of the AI workloads that people do today with frontier models could be done by models that are cheaper, faster, more customizable, more controllable. And they don't do it because frankly it's a pain to pick the right model." "You're subsidized so you don't have to care because you have your subscription, so you route everything to Einstein. 'Hey Einstein, what's the weather today?'" "In normal life, he would be like, 'I'm not answering your silly question.' But because it's AI and subsidized by the AI labs, all the questions are getting routed to Einstein versus in an ideal world you have different models that are more specialized and better at answering your questions in different domains." "It's possible that routing is gonna redistribute a lot of the value capture from frontier models to a more long tail of models. It's like AI maturing... we are in the first phase where it's very simple, everyone using just one giant model. Now we're moving to the second phase."

MTS

50,237 Aufrufe • vor 4 Monaten

China is once again unsettling its American competitors in the AI race. Chinese labs have returned to the spotlight with the unveiling of GLM 5.2, described as the most powerful AI model ever trained in China — and one of the leading open‑source models worldwide. 🔍 Key highlights of “GLM 5.2”: ♦️ It is considered the strongest AI model developed in China to date, and ranks among the smartest open‑source models available. ♦️ It operates at a cost less than one‑tenth that of Anthropic’s latest models, making it a far more economical option. ♦️ It is likely to attract developers and companies seeking powerful, low‑cost models — especially amid growing technical and geopolitical restrictions. ♦️ Some evaluations place the model among the top globally, trailing OpenAI’s models while outperforming several competitors in specialized benchmarks and performance tests.
0:42

China is once again unsettling its American competitors in the AI race. Chinese labs have returned to the spotlight with the unveiling of GLM 5.2, described as the most powerful AI model ever trained in China — and one of the leading open‑source models worldwide. 🔍 Key highlights of “GLM 5.2”: ♦️ It is considered the strongest AI model developed in China to date, and ranks among the smartest open‑source models available. ♦️ It operates at a cost less than one‑tenth that of Anthropic’s latest models, making it a far more economical option. ♦️ It is likely to attract developers and companies seeking powerful, low‑cost models — especially amid growing technical and geopolitical restrictions. ♦️ Some evaluations place the model among the top globally, trailing OpenAI’s models while outperforming several competitors in specialized benchmarks and performance tests.

China pulse 🇨🇳

14,293 Aufrufe • vor 4 Monaten

The Model YL was on display in Naperville, Illinois last weekend. 😍 Illinois was one of the first two locations to display this car to the public… well, at least here in the USA. 😉 This is a short review of the car. I cover some of the features that are different from the Model Y Standard and Premium models. ✅
4:57

The Model YL was on display in Naperville, Illinois last weekend. 😍 Illinois was one of the first two locations to display this car to the public… well, at least here in the USA. 😉 This is a short review of the car. I cover some of the features that are different from the Model Y Standard and Premium models. ✅

TechGeek Tesla 🔋⚡️

36,094 Aufrufe • vor 3 Monaten

I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game). The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies. I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump. Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game. You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs:
18:13

I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game). The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies. I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump. Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game. You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs:

Dwarkesh Patel

610,716 Aufrufe • vor 9 Tagen

There is no best model. There's a lot of noise about models right now. Who is training them, who owns them, where legal intelligence should live. One question actually matters: what produces the best outcome for the legal task in front of you? That's how we decide things at Legora. We optimize for the end-to-end outcome on a legal task. The model is one layer of that system, not the system. Models are uneven and the frontier changes almost weekly. One model plans a long job well, another runs deep analysis across thousands of documents. Some have to be told exactly what to do, and some are fine with a vague brief. They all break in different ways. So our lawyers write evals and we test them with the Legora BAR, our benchmark for agentic reasoning. Every model takes every test, and the model that wins gets the work. We post-train when we know it buys our customers better performance on a specialized task. Training is a tool we reach for when it helps, nothing more than that. The intelligence that compounds sits in the orchestration layer. Precedents, review standards, client requirements. That knowledge has to stay editable, auditable and portable. In our system, a changed review standard is an edit that takes effect the same day, with no new model training required. No lawyer should have to worry about which model did the work, any more than they think about which chip is in their laptop. They should only care about the quality of the work. That's what we are focused on. If you want the engineering version of this argument rather than the CEO version, our CPO, Bryan Tsao, and CTO, Jacob Lauritzen, take it apart in the video below.
9:33

There is no best model. There's a lot of noise about models right now. Who is training them, who owns them, where legal intelligence should live. One question actually matters: what produces the best outcome for the legal task in front of you? That's how we decide things at Legora. We optimize for the end-to-end outcome on a legal task. The model is one layer of that system, not the system. Models are uneven and the frontier changes almost weekly. One model plans a long job well, another runs deep analysis across thousands of documents. Some have to be told exactly what to do, and some are fine with a vague brief. They all break in different ways. So our lawyers write evals and we test them with the Legora BAR, our benchmark for agentic reasoning. Every model takes every test, and the model that wins gets the work. We post-train when we know it buys our customers better performance on a specialized task. Training is a tool we reach for when it helps, nothing more than that. The intelligence that compounds sits in the orchestration layer. Precedents, review standards, client requirements. That knowledge has to stay editable, auditable and portable. In our system, a changed review standard is an edit that takes effect the same day, with no new model training required. No lawyer should have to worry about which model did the work, any more than they think about which chip is in their laptop. They should only care about the quality of the work. That's what we are focused on. If you want the engineering version of this argument rather than the CEO version, our CPO, Bryan Tsao, and CTO, Jacob Lauritzen, take it apart in the video below.

Max Junestrand

49,595 Aufrufe • vor 1 Monat

There's an economics theorem called Alchian-Allen. And it has the very interesting implication that AI labs will be able to charge *higher* margins on their best models as compute gets scarcer. As compute gets more expensive, the cost of running any model goes up. So you might as well pay a bit more to make sure you're running the very best model. Which means the economics of being at the frontier improve, because if you’re not running the very best model, then you’re underutilizing this very precious compute. This pushes the AI model market towards winner-take-all; if you're the best, you can get away with charging an even higher margin. Dylan Patel tells me that we’re already seeing this today: all the revenue in the industry is on the best models. That’s the Alchian-Allen effect. If there’s a cost increase that’s roughly the same for all products, then the relative difference in price between higher and lower quality goods actually goes down. Consumers become relatively more willing to pay for the premium product. And it means that as the compute shortage hits, AI labs can capture more margin - not less, as you might expect - because consumers are choosing premium models more often.
1:21

There's an economics theorem called Alchian-Allen. And it has the very interesting implication that AI labs will be able to charge *higher* margins on their best models as compute gets scarcer. As compute gets more expensive, the cost of running any model goes up. So you might as well pay a bit more to make sure you're running the very best model. Which means the economics of being at the frontier improve, because if you’re not running the very best model, then you’re underutilizing this very precious compute. This pushes the AI model market towards winner-take-all; if you're the best, you can get away with charging an even higher margin. Dylan Patel tells me that we’re already seeing this today: all the revenue in the industry is on the best models. That’s the Alchian-Allen effect. If there’s a cost increase that’s roughly the same for all products, then the relative difference in price between higher and lower quality goods actually goes down. Consumers become relatively more willing to pay for the premium product. And it means that as the compute shortage hits, AI labs can capture more margin - not less, as you might expect - because consumers are choosing premium models more often.

Dwarkesh Patel

323,500 Aufrufe • vor 8 Monaten

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan
3:34

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

MTS

61,449 Aufrufe • vor 3 Monaten

Factory's Taylor Savage argues that Claude Code and Codex aren't necessarily the best tools for their own AI models, and third-party harnesses can actually outperform them: "One thing that I deeply believed before I started at Factory was that the harnesses created by the model companies are going to be the best for their models. Claude Code is going to be the best for Anthropic models, obviously, and Codex is going to be the best for OpenAI models, obviously." "Starting at Factory, I realized that is just not true at all. You can actually do a lot better than those harnesses, certainly on specific types of workloads, but also even in the general case. Different incentives than you have if you’re building a third-party harness." "Keeping them separate also allows you to not get too overfit towards one model. You need to be able to delegate between the different models, not just models from one particular company." "Like Shelby Cobra, you’ve got tuners of cars that can take a car that’s built by Ford and make it work better than it rolled off the factory line. You get things coming out from one OEM, and then you’ve got other people that tune it aftermarket. I think that’s absolutely true with harnesses and models." Factory
2:03

Factory's Taylor Savage argues that Claude Code and Codex aren't necessarily the best tools for their own AI models, and third-party harnesses can actually outperform them: "One thing that I deeply believed before I started at Factory was that the harnesses created by the model companies are going to be the best for their models. Claude Code is going to be the best for Anthropic models, obviously, and Codex is going to be the best for OpenAI models, obviously." "Starting at Factory, I realized that is just not true at all. You can actually do a lot better than those harnesses, certainly on specific types of workloads, but also even in the general case. Different incentives than you have if you’re building a third-party harness." "Keeping them separate also allows you to not get too overfit towards one model. You need to be able to delegate between the different models, not just models from one particular company." "Like Shelby Cobra, you’ve got tuners of cars that can take a car that’s built by Ford and make it work better than it rolled off the factory line. You get things coming out from one OEM, and then you’ve got other people that tune it aftermarket. I think that’s absolutely true with harnesses and models." Factory

MTS

17,436 Aufrufe • vor 1 Monat