正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Sriram Krishnan on why Chinese models can distill off American models but American startups can't: "First of all, every model we have today is distilled off of all human knowledge. If you go back to the original GPT or the original Claude, they all had to derive off crawling... the internet, crawling off all of our blogs and tweets and content. They kind of consume human knowledge to bootstrap that." "I'm going to steal this from Dean Meyer of Sequoia, who had a fantastic post yesterday. The situation which is bad today is that some of these models from other countries can train off American models, whereas if you are an American open weight model, it may be really confusing or challenging on whether you can distill off of other American models." "We kind of have a really uneven ecosystem here, where if you're a Chinese model, you could probably get a bunch of reasoning traces. But if you are a new Valley startup and you want to use some reasoning traces, you don't know what the legal situation is." "If you look at any American open source model today, they are using Chinese models as a teacher, or in a way as a part of the fine-tuning process. How do we make sure that is protected and enshrined, so if you're an American model company, you have the same level playing field as the Chinese models?" Sriram Krishnanshow more

MTS

348,531 subscribers

71,286 次观看 • 1 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now: "I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American." "Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed." "There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in." "The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure." "It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that." Sriram Krishnan

MTS

61,234 次观看 • 1 个月前

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 次观看 • 2 年前

David Sacks: Anthropic Is Trying to Crush Open Source AI and the American Developers Who Use It David Sacks: “I know people don't have a lot of sympathy for Chinese companies, that's fine. I'm not defending Chinese companies. I'm defending American developers who need to be able to use everything in the public domain. And let me give you an example. Cursor rolled out its new product, Composer 2. They were able to post-train that model using Kimi K2.5 on their own proprietary coding data. They started with a Chinese open source model, and then they used their own data, and they came up with a new derivative product. This is the way that open source works. You take things that are in the public domain, you fork them, you make them your own. And by the way, once it's in the public domain, it's not a Chinese model anymore. No data is going back to China, nothing's going back to China. An American company has taken open source contributions in the public domain, made it their own, and then developed their own model. And if you say that American companies can't do that, or that somehow it's tainted with IP theft, you are basically going to put a dagger through the heart of the entire American open source ecosystem. And that is exactly what Anthropic wants, because they do not want to have the competition.”

David Sacks: Anthropic Is Trying to Crush Open Source AI and the American Developers Who Use It David Sacks: “I know people don't have a lot of sympathy for Chinese companies, that's fine. I'm not defending Chinese companies. I'm defending American developers who need to be able to use everything in the public domain. And let me give you an example. Cursor rolled out its new product, Composer 2. They were able to post-train that model using Kimi K2.5 on their own proprietary coding data. They started with a Chinese open source model, and then they used their own data, and they came up with a new derivative product. This is the way that open source works. You take things that are in the public domain, you fork them, you make them your own. And by the way, once it's in the public domain, it's not a Chinese model anymore. No data is going back to China, nothing's going back to China. An American company has taken open source contributions in the public domain, made it their own, and then developed their own model. And if you say that American companies can't do that, or that somehow it's tainted with IP theft, you are basically going to put a dagger through the heart of the entire American open source ecosystem. And that is exactly what Anthropic wants, because they do not want to have the competition.”

The All-In Podcast

277,924 次观看 • 1 个月前

.Anish Acharya says the future of AI isn’t one model to rule them all—and explains why platforms that integrate multiple models will benefit the most: "I think we're going to need and rely on all of the models." "It's sort of like if you have a team of people... if you have five people, they could all do a basic set of things pretty capably." "But then they all have their specializations. Maybe one of them is really good at closing a customer who doesn't want to sign the deal, and one of them is really good at culture and getting the best out of the team." "There are some areas in which they are going to build apps, and that will be a threat to app companies. But there are many areas in which app companies are advantaged. Cursor and Krea are great examples of this—products where you benefit from being multi-model." "When you actually use a creative tool, you don't want to just use Nano Banana, you want to have access to OpenAI, Nano Banana, Kling—all of them—Qwen, you name it. So using a single interface to access all the models is powerful." Anish Acharya on BILLIONS with Guillaume Moubeche

.Anish Acharya says the future of AI isn’t one model to rule them all—and explains why platforms that integrate multiple models will benefit the most: "I think we're going to need and rely on all of the models." "It's sort of like if you have a team of people... if you have five people, they could all do a basic set of things pretty capably." "But then they all have their specializations. Maybe one of them is really good at closing a customer who doesn't want to sign the deal, and one of them is really good at culture and getting the best out of the team." "There are some areas in which they are going to build apps, and that will be a threat to app companies. But there are many areas in which app companies are advantaged. Cursor and Krea are great examples of this—products where you benefit from being multi-model." "When you actually use a creative tool, you don't want to just use Nano Banana, you want to have access to OpenAI, Nano Banana, Kling—all of them—Qwen, you name it. So using a single interface to access all the models is powerful." Anish Acharya on BILLIONS with Guillaume Moubeche

a16z

33,251 次观看 • 5 个月前

Palantir CTO Shyam Sankar says Chinese open source models pose a "threat to American Prosperity": "These Chinese open source models are really the result of distillation attacks." "Most of this is just stolen American IP from frontier labs, and I think the frontier labs should do more to protect that IP, and that's in their own economic interest." "The gravest threat to us is not homicide, it's suicide." "It's an inability to turn these tokens into economic value and generate American prosperity for it." "You see that with the proposed moratorium in New York on data centers. Turning our back on AI would be as consequential a mistake as turning our back on the atom in the '70s." "It's just the beginning, but we're starting to see a burgeoning US open model ecosystem forming." "NVIDIA's Nemotron models are very good." "I think you're going to see in the next month a slate of announcements of other American companies, non-frontier labs, neo labs, who are putting out open weight models that companies are going to be able to cheaply fine-tune and capture their own alpha in weights they control." Via Bloomberg

Palantir CTO Shyam Sankar says Chinese open source models pose a "threat to American Prosperity": "These Chinese open source models are really the result of distillation attacks." "Most of this is just stolen American IP from frontier labs, and I think the frontier labs should do more to protect that IP, and that's in their own economic interest." "The gravest threat to us is not homicide, it's suicide." "It's an inability to turn these tokens into economic value and generate American prosperity for it." "You see that with the proposed moratorium in New York on data centers. Turning our back on AI would be as consequential a mistake as turning our back on the atom in the '70s." "It's just the beginning, but we're starting to see a burgeoning US open model ecosystem forming." "NVIDIA's Nemotron models are very good." "I think you're going to see in the next month a slate of announcements of other American companies, non-frontier labs, neo labs, who are putting out open weight models that companies are going to be able to cheaply fine-tune and capture their own alpha in weights they control." Via Bloomberg

Jawwwn

180,421 次观看 • 1 个月前

Sequoia Partner Dean Meyer says there are two major risks to relying on a Chinese open-source ecosystem: dependency and security. "The first risk is dependency. The framing we have at Sequoia is we're only as good as our next investment. In some sense, the analogy is you're only as good as your next model. So as long as you're dependent on the Chinese to give you the next model, you're in a weird position where you have this dependency." "The other thing is just the risk of backdoors. It's an NP-hard problem to know if there is a backdoor in a model that we know. So it's impossible to verify." "The state of AI security today is very similar to the early days of the Internet when Wi-Fi encryption was publicly broken and no one did anything. I don't even think we're at that moment yet." "It's going to get really bad if we don't better understand how to secure these things and build our own independent ecosystem."

Sequoia Partner Dean Meyer says there are two major risks to relying on a Chinese open-source ecosystem: dependency and security. "The first risk is dependency. The framing we have at Sequoia is we're only as good as our next investment. In some sense, the analogy is you're only as good as your next model. So as long as you're dependent on the Chinese to give you the next model, you're in a weird position where you have this dependency." "The other thing is just the risk of backdoors. It's an NP-hard problem to know if there is a backdoor in a model that we know. So it's impossible to verify." "The state of AI security today is very similar to the early days of the Internet when Wi-Fi encryption was publicly broken and no one did anything. I don't even think we're at that moment yet." "It's going to get really bad if we don't better understand how to secure these things and build our own independent ecosystem."

TBPN

39,809 次观看 • 1 个月前

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

MTS

14,406 次观看 • 2 个月前

Head of Nemotron Bryan Catanzaro says NVIDIA is sitting on internal "teacher" models that beat everything it has publicly shipped and he's trying to release these as well "This technology of using teacher models and multi-domain on-policy distillation has been really key for us in these ways." "Now, what does that mean? Well, it definitely means that inside NVIDIA we have a collection of teacher models that are individually better at different domains than any of the models that we've released." "And actually, I'm trying to release those teacher models as part of the Nemotron project as well -- because as you know, Nemotron is not just a model." "Nemotron is an open technology effort. It includes data sets, it includes techniques, all of our ideas about how to build a model, which we publish." "Of course it includes the models themselves, but I think it should also include collections of teacher models." "I'd imagine this could be a really great way of federating open technologies for AI across the ecosystem." _______ Wrote more on a prior Bryan interview here:

Head of Nemotron Bryan Catanzaro says NVIDIA is sitting on internal "teacher" models that beat everything it has publicly shipped and he's trying to release these as well "This technology of using teacher models and multi-domain on-policy distillation has been really key for us in these ways." "Now, what does that mean? Well, it definitely means that inside NVIDIA we have a collection of teacher models that are individually better at different domains than any of the models that we've released." "And actually, I'm trying to release those teacher models as part of the Nemotron project as well -- because as you know, Nemotron is not just a model." "Nemotron is an open technology effort. It includes data sets, it includes techniques, all of our ideas about how to build a model, which we publish." "Of course it includes the models themselves, but I think it should also include collections of teacher models." "I'd imagine this could be a really great way of federating open technologies for AI across the ecosystem." _______ Wrote more on a prior Bryan interview here:

Fireside Alpha

29,125 次观看 • 5 天前

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

Harry Stebbings

63,738 次观看 • 2 个月前

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Harper Carroll

30,392 次观看 • 26 天前

🚨“The Chinese, who manufacture virtually all of our solar panels…have been installing cellular radios inside the inverters, which can act as kill switches," says Michael Shellenberger "There was actually a Chinese solar company that turned off the solar panels and sent up a little message to its users, saying: we've turned this off because these solar panels are only authorized to be used in certain countries. …So we know that the Chinese solar companies—all of which essentially can be used as arms of the Chinese military…they are able to to switch off large amounts of solar. And if you are able to do that, then you are able to cause the de-powering and blackouts of the grid. How big of a problem is this? …I think most national security experts view this as something that the Chinese could do if there was, for example, an invasion of Taiwan, and the United States decided to defend Taiwan. The Chinese could potentially trigger blackouts in the United States and Europe. As crazy as that sounds, that is the reality."

🚨“The Chinese, who manufacture virtually all of our solar panels…have been installing cellular radios inside the inverters, which can act as kill switches," says Michael Shellenberger "There was actually a Chinese solar company that turned off the solar panels and sent up a little message to its users, saying: we've turned this off because these solar panels are only authorized to be used in certain countries. …So we know that the Chinese solar companies—all of which essentially can be used as arms of the Chinese military…they are able to to switch off large amounts of solar. And if you are able to do that, then you are able to cause the de-powering and blackouts of the grid. How big of a problem is this? …I think most national security experts view this as something that the Chinese could do if there was, for example, an invasion of Taiwan, and the United States decided to defend Taiwan. The Chinese could potentially trigger blackouts in the United States and Europe. As crazy as that sounds, that is the reality."

Jan Jekielek

549,335 次观看 • 1 年前

In between major AI model breakthroughs, new models still ship, but they’re not fundamentally expanding what you can do with them. Braintrust CEO Ankur Goyal says that’s exactly when open source starts to surge: "When new models come out, people forget about open source and forget about economics because it changes fundamentally what you can do." "Programming is a good example. The workflow today is completely different than it was a year ago. If you tried to use models from a year ago today, you wouldn't be able to do crazy stuff like OpenClaw or whatever people are building nowadays." "When models don't change at that speed, people optimize the performance on the use cases that they have." "If you look at usage on our platform, almost half the token usage is coming from open source models from a very small number of companies that have figured out how to optimize use cases really well."

In between major AI model breakthroughs, new models still ship, but they’re not fundamentally expanding what you can do with them. Braintrust CEO Ankur Goyal says that’s exactly when open source starts to surge: "When new models come out, people forget about open source and forget about economics because it changes fundamentally what you can do." "Programming is a good example. The workflow today is completely different than it was a year ago. If you tried to use models from a year ago today, you wouldn't be able to do crazy stuff like OpenClaw or whatever people are building nowadays." "When models don't change at that speed, people optimize the performance on the use cases that they have." "If you look at usage on our platform, almost half the token usage is coming from open source models from a very small number of companies that have figured out how to optimize use cases really well."

TBPN

16,294 次观看 • 6 个月前

Distilled recap of the back-and-forth with Jensen on export controls: Dwarkesh: Wouldn’t selling Nvidia chips to China enable them to train models like Claude Mythos with cyber offensive capabilities that would be threats to American companies and national security? Jensen: First of all, Mythos was trained on fairly mundane capacity and a fairly mundane amount of it by an extraordinary company. The amount of capacity and the type of compute it was trained on is abundantly available in China. Dwarkesh: With that, could they eventually train a model like Mythos? Yes. But the question is, because we have more FLOPs, American labs are able to get to this level of capabilities first. Furthermore, even if they trained a model like this, the ability to deploy it at scale matters. If you had a cyber hacker, it's much more dangerous if they have a million of them versus a thousand of them. Jensen: Your premise is just wrong. The fact of the matter is their AI development is going just fine. The best AI researchers in the world, because they are limited in compute, also come up with extremely smart algorithms. DeepSeek is not an inconsequential advance. The day that DeepSeek comes out on Huawei first, that is a horrible outcome for our nation. Dwarkesh: Currently, you can have a model like DeepSeek that can run on any accelerator if it's open source. Why would that stop being the case in the future? Jensen: Suppose it optimizes for Huawei. Suppose it optimizes for their architecture. It would put others at a disadvantage. As AI diffuses out into the rest of the world, their standards and their tech stack will become superior to ours because their models are open. Dwarkesh: Tesla sold extremely good electric vehicles to China for a long time. iPhones are sold in China. They didn't cause some lock-in. China will still make their version of EVs, and they're dominating, or smartphones, they're dominating. Jensen: We are not a car. The fact that I can buy this car brand one day and use another car brand another day is easy. Computing is not like that. There's a reason why x86 still exists. There's a reason why Arm is so sticky. These ecosystems are hard to replace. Dwarkesh: It's just hard to imagine that there's a long-term lock-in to the Chinese ecosystem, even if they have this slightly better open-source model for a while. American labs port across accelerators constantly. Anthropic's models are run on GPUs, they're run on Trainium, they're run on TPUs. There are so many things you can do, from distilling to a model that's well fit for your chips. Jensen: China is the largest contributor to open source software in the world. China's the largest contributor to open models in the world. Today it's built on the American tech stack, Nvidia’s. Fact. All five layers of the tech stack for AI are important. The United States ought to go win all five of them. in a few years time, I'm making you the prediction that when we want American technology to be diffused around the world—out to India, out to the Middle East, out to Africa, out to Southeast Asia—on that day, I will tell you exactly about today's conversation, about how your policy ... caused the United States to concede the second largest market in the world for no good reason at all.

Dwarkesh Patel

1,252,795 次观看 • 4 个月前

MIT PhD student Alex Zhang admits current AI models might already be AGI with the right harness around them: "To an extent, the answer is kind of yes. The strongest thing I can say is that we can do a lot more with the current models that we have." "Whether that means actually automating things, I'm not actually sure, but one of the obvious things that we find with even harnesses outside of Prime Agent is that harnesses can greatly increase the capabilities and generalization of models." "There's a lot of cases where we think models can't be used or they can't be applied for these settings, but the likely answer is that this actually isn't true. You probably can apply a model to the setting if you have the right harness." alex zhang Seth Karten Prime Intellect

MIT PhD student Alex Zhang admits current AI models might already be AGI with the right harness around them: "To an extent, the answer is kind of yes. The strongest thing I can say is that we can do a lot more with the current models that we have." "Whether that means actually automating things, I'm not actually sure, but one of the obvious things that we find with even harnesses outside of Prime Agent is that harnesses can greatly increase the capabilities and generalization of models." "There's a lot of cases where we think models can't be used or they can't be applied for these settings, but the likely answer is that this actually isn't true. You probably can apply a model to the setting if you have the right harness." alex zhang Seth Karten Prime Intellect

MTS

28,436 次观看 • 24 天前

China just released an open source AI model that matches the best closed models from OpenAI and Anthropic. Gavin Baker explained exactly how they did it and the answer should concern every American AI lab. The model is called GLM 5.2. It was built by Z. AI. You get 744 billion parameters, 1 million token context window and its MIT license, meaning anyone can download it, fork it, build a company on it, with no restrictions and no Dario. It scored 51 points on the artificial analysis intelligence index. The highest score any open weight model has ever achieved. It beat GPT 5.5 on the frontier software engineering benchmark. It trails Claude Opus 4.8 by less than one percentage point. And it costs 85% less to run than GPT 5.5 for comparable performance. Gavin Baker said on the All-In podcast that this model has challenged some of his beliefs. Then he explained how China built it. The method is called distillation. Just think of tens of thousands of phones and computers running simultaneously, all hitting the frontier model APIs through masked accounts, asking specific questions, and harvesting what happens inside the model when it answers. Every reasoning step, every token. The entire thinking process gets recorded and fed back into the Chinese model during training. It is a cheat sheet. It is the answer key to the exam. And here is the part that should worry everyone. Sacks said it plainly. China was already nine months behind American models. But now that GLM 5.2 is good enough to run its own reinforcement learning, it can improve itself without needing to distill from American models anymore. The cheat sheet let them get close enough to start writing their own answers. Sacks said we are six months behind on the model and 24 months behind on silicon and they are only a few months behind in total. The Z. AI founder told Elon Musk directly that open weight fable-level capability will be here before Q1 2027. Every restriction Anthropic lobbied for, every self-imposed safety guardrail, every month of delay in releasing American frontier models accelerated this. The Chinese labs were not under those restrictions. They were not going to wait. The composable model future Gavin described, where every enterprise runs a frontier model alongside their own fine-tuned open weight model, is coming regardless of what American labs do next. The question is just whether the open weight half of that stack is American or Chinese. Right now it is Chinese. WATCH THE FULL PODCAST ON The All-In Podcast

China just released an open source AI model that matches the best closed models from OpenAI and Anthropic. Gavin Baker explained exactly how they did it and the answer should concern every American AI lab. The model is called GLM 5.2. It was built by Z. AI. You get 744 billion parameters, 1 million token context window and its MIT license, meaning anyone can download it, fork it, build a company on it, with no restrictions and no Dario. It scored 51 points on the artificial analysis intelligence index. The highest score any open weight model has ever achieved. It beat GPT 5.5 on the frontier software engineering benchmark. It trails Claude Opus 4.8 by less than one percentage point. And it costs 85% less to run than GPT 5.5 for comparable performance. Gavin Baker said on the All-In podcast that this model has challenged some of his beliefs. Then he explained how China built it. The method is called distillation. Just think of tens of thousands of phones and computers running simultaneously, all hitting the frontier model APIs through masked accounts, asking specific questions, and harvesting what happens inside the model when it answers. Every reasoning step, every token. The entire thinking process gets recorded and fed back into the Chinese model during training. It is a cheat sheet. It is the answer key to the exam. And here is the part that should worry everyone. Sacks said it plainly. China was already nine months behind American models. But now that GLM 5.2 is good enough to run its own reinforcement learning, it can improve itself without needing to distill from American models anymore. The cheat sheet let them get close enough to start writing their own answers. Sacks said we are six months behind on the model and 24 months behind on silicon and they are only a few months behind in total. The Z. AI founder told Elon Musk directly that open weight fable-level capability will be here before Q1 2027. Every restriction Anthropic lobbied for, every self-imposed safety guardrail, every month of delay in releasing American frontier models accelerated this. The Chinese labs were not under those restrictions. They were not going to wait. The composable model future Gavin described, where every enterprise runs a frontier model alongside their own fine-tuned open weight model, is coming regardless of what American labs do next. The question is just whether the open weight half of that stack is American or Chinese. Right now it is Chinese. WATCH THE FULL PODCAST ON The All-In Podcast

Ihtesham Ali

86,452 次观看 • 2 个月前

"Closed models are cheaper. The reason you need open models is that they give you control". - Nvidia CEO Jensen Huang "One of the things that people misunderstand about open models is this: yes, you can host them yourself, and you can build your own computer. But most people use computers in the cloud. Frankly, I think closed models are cheaper. If you do not build everything yourself, you avoid the significant cost and expertise required to train, fine-tune, maintain, guardrail, evaluate, and keep the model safe. You would also need to build or obtain the computers required to host it. So, there is nothing cheap about doing all of that yourself. The reason you need open models is that they give you control. You can adapt them for your own highly specialized use cases. I think there is a lot of misunderstanding about closed models versus open models. We felt it was important for people to understand that there is a place for both." ---- From "Bloomberg Podcasts" YouTube channel, (full video link in comment)

"Closed models are cheaper. The reason you need open models is that they give you control". - Nvidia CEO Jensen Huang "One of the things that people misunderstand about open models is this: yes, you can host them yourself, and you can build your own computer. But most people use computers in the cloud. Frankly, I think closed models are cheaper. If you do not build everything yourself, you avoid the significant cost and expertise required to train, fine-tune, maintain, guardrail, evaluate, and keep the model safe. You would also need to build or obtain the computers required to host it. So, there is nothing cheap about doing all of that yourself. The reason you need open models is that they give you control. You can adapt them for your own highly specialized use cases. I think there is a lot of misunderstanding about closed models versus open models. We felt it was important for people to understand that there is a place for both." ---- From "Bloomberg Podcasts" YouTube channel, (full video link in comment)

Rohan Paul

26,597 次观看 • 26 天前

Sam on Kimi, distillation, and open source: “I have always assumed that there are going to be great cheap models in the world, and we better be the greatest and the cheapest. You get a better deal today, at least at a particular latency, using OpenAI's models than Kimi. We distill our own models. That’s how we make smaller, cheaper models. That’s a very good thing to do. There will clearly be an important place for open source models in the world and people that will want their own weights. We have so much usage of our models that we do not need to be a gigantically high-margin business to be able to afford model training. I would rather people not distill from us, for sure. But this is not in my top ten list of worries.”

Sam on Kimi, distillation, and open source: “I have always assumed that there are going to be great cheap models in the world, and we better be the greatest and the cheapest. You get a better deal today, at least at a particular latency, using OpenAI's models than Kimi. We distill our own models. That’s how we make smaller, cheaper models. That’s a very good thing to do. There will clearly be an important place for open source models in the world and people that will want their own weights. We have so much usage of our models that we do not need to be a gigantically high-margin business to be able to afford model training. I would rather people not distill from us, for sure. But this is not in my top ten list of worries.”

Patrick OShaughnessy

269,663 次观看 • 1 个月前

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."

David Sacks says Anthropic could stop Chinese distillation if they wanted to, but they won't because it slows growth "The way that you know that this whole distillation thing is fake, is because if stopping distillation was their primary objective, Anthropic would push to ban Chinese access to American models, not American access to Chinese models." "They’re the ones in the best position to block it. If industrial scale distillation is a national security threat, they’re the ones who need to stop it, because that is the place where distillation occurs. You have to stop it at the source." "They know that if they KYC their customers, it’ll slow their growth. So instead, what they’re saying is, “Hey, ban our competitors.” If they really think it’s that big a threat, they should use a few points of their 90% gross margins to do that." "It seems to me that this debate is all backwards. The question should be on Anthropic to explain why it’s doing such a bad job, not on the whole American open source ecosystem to be punished for Anthropic’s failure."

dnap

191,260 次观看 • 1 个月前

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

TBPN

85,358 次观看 • 7 个月前