正在加载视频...

视频加载失败

So much discussion about whether we should be using Chinese open-source models in the US. But how could they actually hurt us? Matan Grimberg Matan Grinberg breaks down the biggest risks- 1. Intentional vulnerabilities. A model could contain backdoors, sleeper behavior, or other weaknesses deliberately introduced into the weights,...

17,244 次观看 • 23 天前 •via X (Twitter)

9 条评论

Sabrina Halper 的头像
Sabrina Halper23 天前

Full episode on YouTube:

Droid 的头像
Droid23 天前

@matanSF That's my CEO!

Ashley Jepson 的头像
Ashley Jepson23 天前

@matanSF the ceo aura is palpable

Rick A.F. 的头像
Rick A.F.22 天前

I don't know if this true at all tbh. I use deepseek, I have Codex/OpenAI check it with SOL. I mix up models all the time, they collaborate, they change, they code. I don't think there is any hidden stuff like iin this video. Even sleeper behavior does not make sense to me, these models can be hosted locally and can not act alone?

Vikram Angrish 的头像
Vikram Angrish22 天前

@matanSF Compared to what? Closed source? Total Blackbox?

Sebastian Escarrer 的头像
Sebastian Escarrer22 天前

@matanSF the debate is stuck on whether to use them. companies already are. the real question is whether anyone deploying them can say what's inside

号角 的头像
号角22 天前

@matanSF 这是个傻逼吗? 开源模型的权重全部都是可以审核的,它的架构推理你也可以部署在本地。 比模型权重没有开放,系统提示词不明确的,推理过程也是黑盒的,要好很多吧?

Richa Sharma 的头像
Richa Sharma23 天前

@matanSF The concept of backdoors still fascinates me actually the first time I heard it (was at a party) from @hendrycks . He could be another great podcast guest Sabrina!

sophs b 的头像
sophs b22 天前

@matanSF Intentional backdoors are a plausible risk, but the bigger threat may be subtle, unintentional biases baked into training data that can't be easily audited. Where's the line between acceptable and unacceptable foreign influence?

相关视频

The biggest AI companies may be using safety to lock everyone else out of the future (Save this). David Sacks believes that warnings about catastrophic AI risks will build public support for a powerful federal regulator. That regulator may never explicitly ban open source AI. Instead, it could require every advanced model to remain continuously monitored, centrally controlled and capable of being withdrawn. Closed models such as Claude could satisfy those requirements because they operate on company controlled servers. Open weight models would struggle to comply because anyone can download, copy and modify their underlying weights. Once those weights are publicly released, the developer cannot recall every copy, monitor every user or guarantee that its original safeguards remain intact. Anthropic identifies this irreversibility as a legitimate security concern. Applying identical rules to open and closed models could therefore produce very unequal consequences. Anthropic and OpenAI could afford expensive testing, licensing and monitoring requirements, while startups and independent developers might be unable to comply. The eventual result could be a government protected oligopoly dominated by a few closed model companies. There is evidence supporting part of Sacks’ concern. Anthropic advocates mandatory pre-release testing for every sufficiently powerful model, whether it is open or closed, with evaluations focused on cyber, biological and alignment risks. However, Anthropic explicitly denies supporting a blanket ban on open weight models. The company describes open models without dangerous capabilities as a public good and argues that regulation should depend on demonstrated capabilities rather than whether a model is open or closed. Sacks’ strongest argument is therefore about regulatory consequences because safety organizations could receive stronger protections, politicians could acquire greater authority and dominant AI companies could gain an expensive compliance moat. And open source competitors could gradually be eliminated without the government ever formally announcing a ban.

Milk Road AI

56,257 次观看 • 13 天前

Bloomberg warns that China’s AI price war may make profitability difficult for years. They should be more worried about Wall Street. If the price war continues, the real casualty may not be AI. It may be the entire valuation structure built around it. OpenAI and Anthropic are valued as though frontier intelligence will remain scarce, expensive, and capable of producing software-like margins. Hyperscalers are spending hundreds of billions on infrastructure based on that assumption. Nvidia’s valuation depends on those capital expenditures continuing for years. But Chinese models are driving token prices toward commodity levels. Usage can explode while revenue per token collapses. The servers become busier. The models become cheaper. The profits fail to appear. Nvidia has real revenue and real margins, so it is not the weakest link. But even Nvidia is priced on the belief that today’s extraordinary AI capital expenditure will remain economically justified. If cheaper models, better efficiency, and open weights destroy the expected returns on that infrastructure, Wall Street will not merely reprice OpenAI and Anthropic. It will reprice the entire AI chain. America built the AI bubble on Nvidia. Nvidia built its empire on TSMC. Taiwan built its economic security around TSMC. If Chinese efficiency destroys the economics of brute-force AI, the repricing will not stop in Silicon Valley. It will cross the Pacific and land in Hsinchu. AI may continue transforming the world while the AI bubble collapses. The internet survived 2000. The valuations did not.

𝘊𝘰𝘳𝘳𝘪𝘯𝘦

13,962 次观看 • 1 个月前

Anthropic admitted they built an AI so capable they were scared to release it and the number that explains why is 250. Anthropic's CFO Krishna Rao described in this clip what happened when they ran Mythos against an open source codebase that a previous frontier model had already analyzed. The prior model found 22 security vulnerabilities, Mythos found 250. In the same codebase, that the previous model had already reviewed and flagged as relatively clean. That number, more than 11 times as many vulnerabilities discovered is not just a benchmark improvement, it is a signal that there is an entire layer of software infrastructure that humanity has been operating under the assumption was secure and that assumption may no longer hold. The UK AI Security Institute independently evaluated Mythos Preview and confirmed what the internal numbers suggested. On expert level capture the flag challenges that no model could complete before April 2025, Mythos succeeded 73% of the time and it became the first model ever to complete a complex end-to-end attack range from start to finish, autonomously, without human guidance. The World Economic Forum called this a new security-driven era for AI, the Governor of the Bank of England publicly warned that Anthropic may have found a way to unlock the entire cyber-risk landscape, and the European Central Bank began quietly contacting financial institutions to assess their security posture. The response from Anthropic is what makes this story genuinely important. Rather than shelving the model or publishing it as a standard API release, Rao described a phased approach restricting access to a controlled group, focusing specifically on how the cyber capabilities can be used defensively rather than offensively and treating that framework as a template for how to release powerful but dangerous models in the future. The broader context makes that framing even more significant. AI generated code is already creating ten times more security vulnerabilities than human-written code, 63% of organizations reported experiencing an AI driven cyberattack in the past 12 months, and traditional signature-based security tools were built for a threat model that no longer describes the attack surface companies are defending against. Mythos represents a genuine leap in what autonomous security reasoning can do and it cuts both ways. The model that can find 250 vulnerabilities in a codebase a prior model rated as mostly clean is also, in the wrong hands, the model that can exploit those 250 vulnerabilities before a human defender has even finished reading the report. Anthropic's phased release strategy is not just a legal or PR decision, it is the most honest signal yet from a frontier lab that safety governance and capability development can no longer be treated as separate workstreams. The question is not whether this technology gets deployed, it is whether the institutions using it defensively stay ahead of the ones who will eventually use it offensively and whether the labs building it can keep those two timelines from inverting.

Milk Road AI

24,356 次观看 • 4 个月前