Загрузка видео...

Не удалось загрузить видео

На главную

So much discussion about whether we should be using Chinese open-source models in the US. But how could they actually hurt us? Matan Grimberg Matan Grinberg breaks down the biggest risks- 1. Intentional vulnerabilities. A model could contain backdoors, sleeper behavior, or other weaknesses deliberately introduced into the weights,...

17,244 просмотров • 2 месяцев назад •via X (Twitter)

Комментарии: 9

Фото профиля Sabrina Halper
Sabrina Halper2 месяцев назад

Full episode on YouTube:

Фото профиля Droid
Droid2 месяцев назад

@matanSF That's my CEO!

Фото профиля Ashley Jepson
Ashley Jepson2 месяцев назад

@matanSF the ceo aura is palpable

Фото профиля Rick A.F.
Rick A.F.2 месяцев назад

I don't know if this true at all tbh. I use deepseek, I have Codex/OpenAI check it with SOL. I mix up models all the time, they collaborate, they change, they code. I don't think there is any hidden stuff like iin this video. Even sleeper behavior does not make sense to me, these models can be hosted locally and can not act alone?

Фото профиля Vikram Angrish
Vikram Angrish2 месяцев назад

@matanSF Compared to what? Closed source? Total Blackbox?

Фото профиля Sebastian Escarrer
Sebastian Escarrer2 месяцев назад

@matanSF the debate is stuck on whether to use them. companies already are. the real question is whether anyone deploying them can say what's inside

Фото профиля 号角
号角2 месяцев назад

@matanSF 这是个傻逼吗? 开源模型的权重全部都是可以审核的,它的架构推理你也可以部署在本地。 比模型权重没有开放,系统提示词不明确的,推理过程也是黑盒的,要好很多吧?

Фото профиля Richa Sharma
Richa Sharma2 месяцев назад

@matanSF The concept of backdoors still fascinates me actually the first time I heard it (was at a party) from @hendrycks . He could be another great podcast guest Sabrina!

Фото профиля sophs b
sophs b2 месяцев назад

@matanSF Intentional backdoors are a plausible risk, but the bigger threat may be subtle, unintentional biases baked into training data that can't be easily audited. Where's the line between acceptable and unacceptable foreign influence?

Похожие видео

The biggest AI companies may be using safety to lock everyone else out of the future (Save this). David Sacks believes that warnings about catastrophic AI risks will build public support for a powerful federal regulator. That regulator may never explicitly ban open source AI. Instead, it could require every advanced model to remain continuously monitored, centrally controlled and capable of being withdrawn. Closed models such as Claude could satisfy those requirements because they operate on company controlled servers. Open weight models would struggle to comply because anyone can download, copy and modify their underlying weights. Once those weights are publicly released, the developer cannot recall every copy, monitor every user or guarantee that its original safeguards remain intact. Anthropic identifies this irreversibility as a legitimate security concern. Applying identical rules to open and closed models could therefore produce very unequal consequences. Anthropic and OpenAI could afford expensive testing, licensing and monitoring requirements, while startups and independent developers might be unable to comply. The eventual result could be a government protected oligopoly dominated by a few closed model companies. There is evidence supporting part of Sacks’ concern. Anthropic advocates mandatory pre-release testing for every sufficiently powerful model, whether it is open or closed, with evaluations focused on cyber, biological and alignment risks. However, Anthropic explicitly denies supporting a blanket ban on open weight models. The company describes open models without dangerous capabilities as a public good and argues that regulation should depend on demonstrated capabilities rather than whether a model is open or closed. Sacks’ strongest argument is therefore about regulatory consequences because safety organizations could receive stronger protections, politicians could acquire greater authority and dominant AI companies could gain an expensive compliance moat. And open source competitors could gradually be eliminated without the government ever formally announcing a ban.

Milk Road AI

57,724 просмотров • 2 месяцев назад