Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Mark Zuckerberg explains the 405B teacher-model flywheel that could make one giant AI the wrong end state "People are gonna wanna do inference directly on the 405 because it's, you know, by our estimates, it's gonna be about 50% cheaper, I think, than GPT-4o to do that directly." "Because... it's open weights, the ability to take the model and distill it down to whatever size that you want, to use it for synthetic data generation, to use it as a teacher model." "Our vision is that there should be lots of different models. I think every startup out there, every enterprise, governments, they all kind of wanna have their own custom models." "Right now, as open source basically closes the gap, I think you're just gonna see this wide proliferation of models where people now have the incentive to basically customize and build and train exactly the right size model for what they're doing, train their data into it." "They're gonna have the tools to do it because of a lot of the partner integrations that the companies like Amazon are doing with AWS or Databricks or different folks like that who are building these whole suites of services for distilling and fine-tuning open models." The counterintuitive edge is that the 405B model may be most valuable as raw material, not an endpoint. The open model compresses into the right size, absorbs proprietary data, and turns one frontier release into thousands of company-specific systems. Distribution of intelligence beats centralization. - Mark Zuckerberg (Mark Zuckerberg), CEO of Meta, with Rowan Cheungshow more

Karl Mehta

144,023 subscribers

523,111 views • 28 days ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

David Sacks says companies are trapped paying OpenAI & Anthropic because they can't figure out how to use open source models "I think enterprise CTOs would like to shift their token consumption to cheaper models for the obvious reason that it would be more efficient. They are seeing compute costs or token costs skyrocket right now, so everyone's trying to figure this out." "You also have the AI sovereignty issue that Alex Karp talked about. They're worried about giving up the secret sauce or the alpha in their business to a frontier lab that may one day be competing with them. "The problem is, I think in most cases, they don't have the technical ability to do it. Coinbase figured out how to do it. DoorDash figured out how to do it. They built a token routing system that allows them to send frontier tasks to frontier models and non frontier tasks to more mundane models. But I don't think your average enterprise has the technical capability to do that." "This is why the share of wallet of closed models, it actually increased. I think that open source went from 19% last year to 11% this year. So open source as a share of enterprise spending is actually decreasing." "I don't think that means usage is decreasing. I think usage is skyrocketing. It also may be the case that because the whole point of using an open model is you just pay for the compute costs, you don't have to pay a lab, so it may be that it's hard to measure that usage in terms of spend." "But nonetheless, anyone who's saying that these closed models are going to lose or are somehow losing, you're just not seeing it in the data."

David Sacks says companies are trapped paying OpenAI & Anthropic because they can't figure out how to use open source models "I think enterprise CTOs would like to shift their token consumption to cheaper models for the obvious reason that it would be more efficient. They are seeing compute costs or token costs skyrocket right now, so everyone's trying to figure this out." "You also have the AI sovereignty issue that Alex Karp talked about. They're worried about giving up the secret sauce or the alpha in their business to a frontier lab that may one day be competing with them. "The problem is, I think in most cases, they don't have the technical ability to do it. Coinbase figured out how to do it. DoorDash figured out how to do it. They built a token routing system that allows them to send frontier tasks to frontier models and non frontier tasks to more mundane models. But I don't think your average enterprise has the technical capability to do that." "This is why the share of wallet of closed models, it actually increased. I think that open source went from 19% last year to 11% this year. So open source as a share of enterprise spending is actually decreasing." "I don't think that means usage is decreasing. I think usage is skyrocketing. It also may be the case that because the whole point of using an open model is you just pay for the compute costs, you don't have to pay a lab, so it may be that it's hard to measure that usage in terms of spend." "But nonetheless, anyone who's saying that these closed models are going to lose or are somehow losing, you're just not seeing it in the data."

dnap

110,354 views • 29 days ago

Jason Calacanis: Anthropic, OpenAI and others are trying to kill OpenClaw Why? Because an open source agent platform is an existential threat to frontier model companies. @jason: “ People are going to say I'm a conspiracy theorist, but the number one goal, I believe, in the large language model, frontier model space, is to kill (OpenClaw). This is a giant movement to stop it, because this is the equivalent of having an open source Android-like player in the market, and that could be incredibly disruptive. Because, I believe, open source is going to win the day on the large language models and take 90% of the token usage, and I think the entire frontier model space could be undercut by open source. And I think they realize that SLMs, the smaller language models that are verticalized now, that will run on desktops and laptops and are even starting to run on the top ones, that is their biggest competitive threat, and I hope it happens.”

Jason Calacanis: Anthropic, OpenAI and others are trying to kill OpenClaw Why? Because an open source agent platform is an existential threat to frontier model companies. @jason: “ People are going to say I'm a conspiracy theorist, but the number one goal, I believe, in the large language model, frontier model space, is to kill (OpenClaw). This is a giant movement to stop it, because this is the equivalent of having an open source Android-like player in the market, and that could be incredibly disruptive. Because, I believe, open source is going to win the day on the large language models and take 90% of the token usage, and I think the entire frontier model space could be undercut by open source. And I think they realize that SLMs, the smaller language models that are verticalized now, that will run on desktops and laptops and are even starting to run on the top ones, that is their biggest competitive threat, and I hope it happens.”

The All-In Podcast

64,611 views • 4 months ago

“AI MODELS COULD BECOME 10× SMALLER WITHOUT LOSING MOST OF THEIR INTELLIGENCE.” Mark Zuckerberg says one of the most powerful developments in AI is model distillation: taking the capabilities of a much larger model and transferring them into a much smaller one. - Meta's huge frontier models can serve as "teachers" for smaller models. - Zuckerberg says a model could potentially retain around 90–95% of the larger model's intelligence while being only ~10% of its size. - That means dramatically lower inference costs, memory requirements and computing power. - Developers can distill and customize these capabilities for their own applications. And that's where open-source models come into the picture.

“AI MODELS COULD BECOME 10× SMALLER WITHOUT LOSING MOST OF THEIR INTELLIGENCE.” Mark Zuckerberg says one of the most powerful developments in AI is model distillation: taking the capabilities of a much larger model and transferring them into a much smaller one. - Meta's huge frontier models can serve as "teachers" for smaller models. - Zuckerberg says a model could potentially retain around 90–95% of the larger model's intelligence while being only ~10% of its size. - That means dramatically lower inference costs, memory requirements and computing power. - Developers can distill and customize these capabilities for their own applications. And that's where open-source models come into the picture.

SciTech Era

55,915 views • 1 day ago

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

girish

457,932 views • 2 months ago

.Decagon co-founder and CEO Jesse Zhang says enterprise open-source adoption is falling right now, and that's a sign of adoption rather than rejection: "Even though there's a lot of hype for open-source, the share of open-source inference is actually going down right now." "People are spinning up all these new use cases, and if you're spinning up new use cases, of course you're gonna use the frontier models until they're working." "Once it's at that point, they're heavily incentivized to use open-source because it's way cheaper and faster." "Enterprises have a lot of desire to move, but they can only do so many use cases at once. There is inertia there, and they have to go through all the model risk governance and all the security things." "I think it'll take time, but it will get there." Jesse Zhang

.Decagon co-founder and CEO Jesse Zhang says enterprise open-source adoption is falling right now, and that's a sign of adoption rather than rejection: "Even though there's a lot of hype for open-source, the share of open-source inference is actually going down right now." "People are spinning up all these new use cases, and if you're spinning up new use cases, of course you're gonna use the frontier models until they're working." "Once it's at that point, they're heavily incentivized to use open-source because it's way cheaper and faster." "Enterprises have a lot of desire to move, but they can only do so many use cases at once. There is inertia there, and they have to go through all the model risk governance and all the security things." "I think it'll take time, but it will get there." Jesse Zhang

a16z

35,742 views • 10 days ago

$Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.$

Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.

Milk Road AI

27,908 views • 1 month ago

SITUATION EXPLAINED: 70% of frontier model queries could run locally for free. clem 🤗, co-founder and CEO of Hugging Face: "There was an interesting study from Stanford published last year showing that 70% of the queries that people ask to ChatGPT could be accurately answered locally on your laptop. For free." "Most of the AI workloads that people do today with frontier models could be done by models that are cheaper, faster, more customizable, more controllable. And they don't do it because frankly it's a pain to pick the right model." "You're subsidized so you don't have to care because you have your subscription, so you route everything to Einstein. 'Hey Einstein, what's the weather today?'" "In normal life, he would be like, 'I'm not answering your silly question.' But because it's AI and subsidized by the AI labs, all the questions are getting routed to Einstein versus in an ideal world you have different models that are more specialized and better at answering your questions in different domains." "It's possible that routing is gonna redistribute a lot of the value capture from frontier models to a more long tail of models. It's like AI maturing... we are in the first phase where it's very simple, everyone using just one giant model. Now we're moving to the second phase."

SITUATION EXPLAINED: 70% of frontier model queries could run locally for free. clem 🤗, co-founder and CEO of Hugging Face: "There was an interesting study from Stanford published last year showing that 70% of the queries that people ask to ChatGPT could be accurately answered locally on your laptop. For free." "Most of the AI workloads that people do today with frontier models could be done by models that are cheaper, faster, more customizable, more controllable. And they don't do it because frankly it's a pain to pick the right model." "You're subsidized so you don't have to care because you have your subscription, so you route everything to Einstein. 'Hey Einstein, what's the weather today?'" "In normal life, he would be like, 'I'm not answering your silly question.' But because it's AI and subsidized by the AI labs, all the questions are getting routed to Einstein versus in an ideal world you have different models that are more specialized and better at answering your questions in different domains." "It's possible that routing is gonna redistribute a lot of the value capture from frontier models to a more long tail of models. It's like AI maturing... we are in the first phase where it's very simple, everyone using just one giant model. Now we're moving to the second phase."

MTS

49,884 views • 1 month ago

David Sacks: Anthropic Is Trying to Crush Open Source AI and the American Developers Who Use It David Sacks: “I know people don't have a lot of sympathy for Chinese companies, that's fine. I'm not defending Chinese companies. I'm defending American developers who need to be able to use everything in the public domain. And let me give you an example. Cursor rolled out its new product, Composer 2. They were able to post-train that model using Kimi K2.5 on their own proprietary coding data. They started with a Chinese open source model, and then they used their own data, and they came up with a new derivative product. This is the way that open source works. You take things that are in the public domain, you fork them, you make them your own. And by the way, once it's in the public domain, it's not a Chinese model anymore. No data is going back to China, nothing's going back to China. An American company has taken open source contributions in the public domain, made it their own, and then developed their own model. And if you say that American companies can't do that, or that somehow it's tainted with IP theft, you are basically going to put a dagger through the heart of the entire American open source ecosystem. And that is exactly what Anthropic wants, because they do not want to have the competition.”

David Sacks: Anthropic Is Trying to Crush Open Source AI and the American Developers Who Use It David Sacks: “I know people don't have a lot of sympathy for Chinese companies, that's fine. I'm not defending Chinese companies. I'm defending American developers who need to be able to use everything in the public domain. And let me give you an example. Cursor rolled out its new product, Composer 2. They were able to post-train that model using Kimi K2.5 on their own proprietary coding data. They started with a Chinese open source model, and then they used their own data, and they came up with a new derivative product. This is the way that open source works. You take things that are in the public domain, you fork them, you make them your own. And by the way, once it's in the public domain, it's not a Chinese model anymore. No data is going back to China, nothing's going back to China. An American company has taken open source contributions in the public domain, made it their own, and then developed their own model. And if you say that American companies can't do that, or that somehow it's tainted with IP theft, you are basically going to put a dagger through the heart of the entire American open source ecosystem. And that is exactly what Anthropic wants, because they do not want to have the competition.”

The All-In Podcast

277,302 views • 16 days ago

Palantir CTO Shyam Sankar says Chinese open source models pose a "threat to American Prosperity": "These Chinese open source models are really the result of distillation attacks." "Most of this is just stolen American IP from frontier labs, and I think the frontier labs should do more to protect that IP, and that's in their own economic interest." "The gravest threat to us is not homicide, it's suicide." "It's an inability to turn these tokens into economic value and generate American prosperity for it." "You see that with the proposed moratorium in New York on data centers. Turning our back on AI would be as consequential a mistake as turning our back on the atom in the '70s." "It's just the beginning, but we're starting to see a burgeoning US open model ecosystem forming." "NVIDIA's Nemotron models are very good." "I think you're going to see in the next month a slate of announcements of other American companies, non-frontier labs, neo labs, who are putting out open weight models that companies are going to be able to cheaply fine-tune and capture their own alpha in weights they control." Via Bloomberg

Palantir CTO Shyam Sankar says Chinese open source models pose a "threat to American Prosperity": "These Chinese open source models are really the result of distillation attacks." "Most of this is just stolen American IP from frontier labs, and I think the frontier labs should do more to protect that IP, and that's in their own economic interest." "The gravest threat to us is not homicide, it's suicide." "It's an inability to turn these tokens into economic value and generate American prosperity for it." "You see that with the proposed moratorium in New York on data centers. Turning our back on AI would be as consequential a mistake as turning our back on the atom in the '70s." "It's just the beginning, but we're starting to see a burgeoning US open model ecosystem forming." "NVIDIA's Nemotron models are very good." "I think you're going to see in the next month a slate of announcements of other American companies, non-frontier labs, neo labs, who are putting out open weight models that companies are going to be able to cheaply fine-tune and capture their own alpha in weights they control." Via Bloomberg

Jawwwn

180,406 views • 23 days ago

.Josh Wolfe: Anybody Using DeepSeek App Is 'Absolute Fool' "Anybody using the DeepSeek app is an absolute fool. If you're using DeepSeek on companies like Together Compute, one of Lux's companies, which can get rid of the CCP censorship, then it's probably okay. But remember, the open-source movement is something we deeply believe in. Most great technologists, entrepreneurs, and venture capitalists are on the side of open source. The closed-source models that have consumed tens of billions of dollars are the ones that are really going to be at risk. When you look at Hugging Face, a major repository, or Together Compute, Runway ML, and a lot of Lux's companies, they have been pioneers in open source. Now, why am I not worried about open source, even with the DeepSeek model? As long as you don't have the CCP censorship on it, the models with their open weights allow people to run on their proprietary data. This means companies like pharma or defense companies that have their own siloed, proprietary data—think about Bloomberg with their proprietary longitudinal data, or Meta with their data—are the ones who will have the edge. Even as open source takes hold, these companies will still dominate. I’m not worried about open source being the problem. I’m more concerned about people overfunding closed models with no proprietary source. A lot of capital is going to be burned there, and we’re already seeing that with people worried about OpenAI in some aspects."

.Josh Wolfe: Anybody Using DeepSeek App Is 'Absolute Fool' "Anybody using the DeepSeek app is an absolute fool. If you're using DeepSeek on companies like Together Compute, one of Lux's companies, which can get rid of the CCP censorship, then it's probably okay. But remember, the open-source movement is something we deeply believe in. Most great technologists, entrepreneurs, and venture capitalists are on the side of open source. The closed-source models that have consumed tens of billions of dollars are the ones that are really going to be at risk. When you look at Hugging Face, a major repository, or Together Compute, Runway ML, and a lot of Lux's companies, they have been pioneers in open source. Now, why am I not worried about open source, even with the DeepSeek model? As long as you don't have the CCP censorship on it, the models with their open weights allow people to run on their proprietary data. This means companies like pharma or defense companies that have their own siloed, proprietary data—think about Bloomberg with their proprietary longitudinal data, or Meta with their data—are the ones who will have the edge. Even as open source takes hold, these companies will still dominate. I’m not worried about open source being the problem. I’m more concerned about people overfunding closed models with no proprietary source. A lot of capital is going to be burned there, and we’re already seeing that with people worried about OpenAI in some aspects."

Josh Caplan

39,985 views • 1 year ago

"Closed models are cheaper. The reason you need open models is that they give you control". - Nvidia CEO Jensen Huang "One of the things that people misunderstand about open models is this: yes, you can host them yourself, and you can build your own computer. But most people use computers in the cloud. Frankly, I think closed models are cheaper. If you do not build everything yourself, you avoid the significant cost and expertise required to train, fine-tune, maintain, guardrail, evaluate, and keep the model safe. You would also need to build or obtain the computers required to host it. So, there is nothing cheap about doing all of that yourself. The reason you need open models is that they give you control. You can adapt them for your own highly specialized use cases. I think there is a lot of misunderstanding about closed models versus open models. We felt it was important for people to understand that there is a place for both." ---- From "Bloomberg Podcasts" YouTube channel, (full video link in comment)

"Closed models are cheaper. The reason you need open models is that they give you control". - Nvidia CEO Jensen Huang "One of the things that people misunderstand about open models is this: yes, you can host them yourself, and you can build your own computer. But most people use computers in the cloud. Frankly, I think closed models are cheaper. If you do not build everything yourself, you avoid the significant cost and expertise required to train, fine-tune, maintain, guardrail, evaluate, and keep the model safe. You would also need to build or obtain the computers required to host it. So, there is nothing cheap about doing all of that yourself. The reason you need open models is that they give you control. You can adapt them for your own highly specialized use cases. I think there is a lot of misunderstanding about closed models versus open models. We felt it was important for people to understand that there is a place for both." ---- From "Bloomberg Podcasts" YouTube channel, (full video link in comment)

Rohan Paul

25,801 views • 6 days ago

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

Dwarkesh Patel

502,249 views • 8 months ago

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

MTS

14,406 views • 1 month ago

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnan

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnan

MTS

71,286 views • 21 days ago

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Whole Mars Catalog

43,772 views • 6 days ago

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

MTS

61,234 views • 21 days ago

"As roboticists and computer vision people [outside of big tech], do we just wait for the next foundation model?" I share the frustration. It's disempowering when most major progress recently is downstream of "foundation models" that you don't have the compute or data to train.

"As roboticists and computer vision people [outside of big tech], do we just wait for the next foundation model?" I share the frustration. It's disempowering when most major progress recently is downstream of "foundation models" that you don't have the compute or data to train.

Chris Offner

68,742 views • 11 months ago

I think the fact is there are only going to be four or five of these companies that are able to raise the billions required to build the infrastructure to support these models. It's a global competition, and particularly against the Chinese right now because they're trying to beat us on AI, because the country with the best AI is gonna win all the wars. That's what's gonna happen. I think at the end of the day you want diversification in your portfolio, not just one stack. For me, I'm playing the picks and shovels. I'm building energy because you need energy for this stuff, and I'm building data centers. Now, there are different things, but right now we need more electrical power in this country beyond just data centers. We just need power because we got a booming economy.

I think the fact is there are only going to be four or five of these companies that are able to raise the billions required to build the infrastructure to support these models. It's a global competition, and particularly against the Chinese right now because they're trying to beat us on AI, because the country with the best AI is gonna win all the wars. That's what's gonna happen. I think at the end of the day you want diversification in your portfolio, not just one stack. For me, I'm playing the picks and shovels. I'm building energy because you need energy for this stuff, and I'm building data centers. Now, there are different things, but right now we need more electrical power in this country beyond just data centers. We just need power because we got a booming economy.

Kevin O'Leary aka Mr. Wonderful

13,143 views • 2 months ago

I think $PLTR has been getting hit because investors are worried about competition from OpenAI and Anthropic but I’ve been been adding to my position because I think a lot of people have this backwards. LLMs are obviously extremely valuable but as the major models continue to improve, they're also starting to cluster together in performance and because they're generally non-deterministic it means the real enterprise challenge becomes more about making sure that answer is grounded in the right data, tied to the right workflow, approved by the right human and executed in the right operational system. I truly think the concern that OpenAI and Anthropic could move into Palantir’s market is more modest than people assume. Model companies are focused on the model and building intelligence while Palantir is trying to help enterprises turn that intelligence into operational outcomes (very different lanes).

I think $PLTR has been getting hit because investors are worried about competition from OpenAI and Anthropic but I’ve been been adding to my position because I think a lot of people have this backwards. LLMs are obviously extremely valuable but as the major models continue to improve, they're also starting to cluster together in performance and because they're generally non-deterministic it means the real enterprise challenge becomes more about making sure that answer is grounded in the right data, tied to the right workflow, approved by the right human and executed in the right operational system. I truly think the concern that OpenAI and Anthropic could move into Palantir’s market is more modest than people assume. Model companies are focused on the model and building intelligence while Palantir is trying to help enterprises turn that intelligence into operational outcomes (very different lanes).

Shay Boloor

428,259 views • 1 month ago

David Friedberg says Anthropic asked big pharma for their data and nearly everyone said no "There's been an effort by Anthropic to sign up life sciences companies to contribute to a new life sciences focused model. They're approaching these large companies with large proprietary data sets and saying, if you share your data, we will give you early access, some sort of proprietary value. Sign this NDA and you can participate with us." "I think nearly everyone I've spoken with has woken up to the fact that they are trying to commoditize everyone's business. If all of the tens of billions of dollars you have invested in experiments and product development, and you've generated all of this proprietary data along the way, that data is a true asset of your organization. It's an asset that you've spent billions of dollars developing." "And by handing it over to a model company to then combine with other people's data, you are commoditizing the one core differentiation that you have. And so everyone is largely saying no." "I think what everyone's realizing is they're better off developing their own weights and their own models using either an open source basis or there might be some intermediary business model that evolves."

David Friedberg says Anthropic asked big pharma for their data and nearly everyone said no "There's been an effort by Anthropic to sign up life sciences companies to contribute to a new life sciences focused model. They're approaching these large companies with large proprietary data sets and saying, if you share your data, we will give you early access, some sort of proprietary value. Sign this NDA and you can participate with us." "I think nearly everyone I've spoken with has woken up to the fact that they are trying to commoditize everyone's business. If all of the tens of billions of dollars you have invested in experiments and product development, and you've generated all of this proprietary data along the way, that data is a true asset of your organization. It's an asset that you've spent billions of dollars developing." "And by handing it over to a model company to then combine with other people's data, you are commoditizing the one core differentiation that you have. And so everyone is largely saying no." "I think what everyone's realizing is they're better off developing their own weights and their own models using either an open source basis or there might be some intermediary business model that evolves."

dnap

286,540 views • 1 month ago