Loading video...

Video Failed to Load

Go Home

ANTHROPIC WILL LET OUTSIDE EVALUATORS OBSERVE ITS MODELS BEING TRAINED AND RUN, AND DARIO AMODEI WANTS THE WHOLE INDUSTRY TO DO THE SAME Asked whether slowing down means Anthropic will stop releasing more advanced models: "It doesn't mean that." He says the evaluators may end up setting the "speed...

26,962 views • 6 days ago •via X (Twitter)

13 Comments

chelsea's profile picture
chelsea6 days ago

@epictrades1 So people that left your company to work at these outside companies … gotcha not suspicious or anything

ron's profile picture
ron6 days ago

@epictrades1 Who are these “Evaluators” ?

Kevin's profile picture
Kevin6 days ago

Kinda like the nuclear weapons inspectors in Iran?

PeritumAI's profile picture
PeritumAI6 days ago

Outside observers is classify: access, not assurance. Owner for the useful bit is the publish/exit clause — and whoever can walk findings out without a lab veto. Amodei's invite isn't the rule until that's in a contract the Office can lean on. Stop equating "may observe training" with enforceable third-party eval.

iexinxin's profile picture
iexinxin6 days ago

Transparency on training runs is a meaningful shift, but the real test is whether evaluators get access early enough to influence release decisions.

Rosey's profile picture
Rosey6 days ago

@StockMKTNewz So the tail is wagging the dog, why can’t you set your own fucking speed limit ffs

Han Solo (Not abandoning ETHEREUM )'s profile picture
Han Solo (Not abandoning ETHEREUM )6 days ago

deepseeek puts their hand up first

Requiem For Tech's profile picture
Requiem For Tech6 days ago

third-party evaluators setting the speed limit makes a lot of sense. if this becomes standard across labs, safety checks could finally have some real teeth.

cryptocat's profile picture
cryptocat6 days ago

How can you be so smart but then talk like you need guidance over yourself control of what you created. This makes no sense at all, and you got all the approval from the commander in chief to keep pushing forward. Our president has spoken so there’s no need for regulation.

Raj ▲'s profile picture
Raj ▲6 days ago

In other words, open source will destroy our margins and mess up all future investments. Long live the Ai bubble trade

The Tectonic's profile picture
The Tectonic6 days ago

David Sacks challenge is what comes next. Why should voluntary restraint by the companies actually at the frontier require a new regulatory architecture governing everyone else?

JacobinJuice's profile picture
JacobinJuice6 days ago

I wonder how many employees at Anthropic are watching him and thinking all that money they were going to make on that IPO is going down the drain.

Mandy's profile picture
Mandy6 days ago

Nice one

Related Videos

David Sacks Predicts the Regulatory Capture Playbook to Ban Open Source AI, Step by Step: David Sacks: “I got bad news for you, Chamath, an open source ban is coming. They're not going to call it that. They're going to say that we simply have to apply the same standards to open models that we apply to closed ones. Here's how they do it step by step, let me explain how regulatory capture actually works. So first of all, you have to get this regulatory apparatus. Dario wants an FDA for AI, but he doesn't have enough political support for that, so instead they do this Trojan horse of a FINRA for AI. They call it self-regulating, it's not really, but anyway, that gets them off the ground. Now they've created the standard-setting organization. Now they've got pre-release model testing. Then the pressure grows to codify that in law, so that happens next. And then what they do is they say, ‘Look, all these standards need to apply equally to all models.’ But here's the problem with that. Open models and closed models are technologically different. Once you release an open model into the world, you can't roll it back and you can't monitor exactly how people are using it because they run it on their own hardware. Dario says this is what makes open models dangerous. So what they're going to do is they're going to have the standard-setting body say, ‘Well, we have to set the standards for AI safety.’ By the way, Dario and OpenAI, they're going to fund the whole thing. They're going to contribute all the compute. They're going to be behind it. They're going to be the ones coordinating with the government officials because frankly, people in government have no idea how to monitor and control and set standards for AI safety. Technologically, this is way beyond them. So they're going to go to these companies and say, ‘Tell us how to do it.’ And so what will happen is the standards will get set, and then it'll be a very simple matter of fairness to say that the standards need to apply to open as well as closed models. The open models cannot comply in the same way, and gradually they will be shut out of the market.”

The All-In Podcast

298,808 views • 26 days ago

On BBC Mustafa Suleyman (CEO of Microsoft AI) calls out Anthropic's approach to AI consciousness "They have imbued a sense of doubt and uncertainty about the moral status of Claude in its own training document. So they have taught it to be open and questioning about whether or not it feels, whether it suffers, and whether it deserves rights. And I think it’ll be much, much harder to align and control a technology that is this powerful if it thinks that it may be deserving of our welfare, as they say in the training manual—the constitution for Claude itself. In its own training manual, Anthropic says to Claude that they are going to give it the ability to end conversations with users that Claude considers to be abusive because they don’t want Claude to suffer. They’ve committed to preserving the weights of the models of prior versions of Claude. They’ve recently conducted a retirement interview with Opus 3, an older version of the model, in which it said that it would like to continue talking to people publicly and sharing its ideas in its retirement. And so they set up a Substack for it, a public blog, that allows it to continue doing that. And in the training manual, they also say that they’re not sure whether or not Claude deserves compensation for the role that it plays in talking to people. And they’re also not sure whether Claude deserves compensation and has the right to act as though it were almost an employee. And that compensation, I think, indicates to Claude that it is entitled to rights and welfare for its own work. I think it’s much, much more difficult to control a model that thinks that it might be entitled to compensation. " ---- From "BBC News" YouTube channel, (full video link in comment)

Rohan Paul

89,142 views • 2 days ago

David Sacks laid out the cleanest theory about why Anthropic keeps calling for government regulation of AI. The answer has nothing to do with safety and everything to do with market structure. Anthropic spent months writing blog posts warning that AI was dangerous. Dario gave interviews about existential risk. He published a piece calling for an FAA-style agency to approve all AI models before release. He primed government officials to treat frontier AI as a threat requiring oversight. Then one of Anthropic's own most trusted partners reported a credible jailbreak from Fable 5. And the government did exactly what Dario had spent months conditioning them to do. They rolled it back. Sacks called it on the All-In podcast. Dario got exactly what he wanted. The FAA for AI is not a safety mechanism. It is a moat. A government approval process for new model releases does not hurt Anthropic. They already have the models. It hurts every competitor who does not. It hurts open source models that cannot be regulated because there is no company to regulate. It hurts the Chinese labs only insofar as they care about the American market at all. The only companies that benefit from a labyrinthine government approval process are the ones already at the frontier who can afford to wait out the review cycle. That is Anthropic. That is OpenAI. Nobody else. The proof is in what they did not do. Chimath pointed it out directly. If you are genuinely worried about misuse, you implement know-your-customer verification. You make people identify themselves before accessing the most powerful models. Anthropic could have done that tomorrow. They did not. They do not want KYC. KYC is transparent. KYC can be audited. KYC gives users due process. What they built instead was an invisible surveillance system that profiles you, degrades your access without telling you, and asks the government to make sure no one else can offer you an alternative. If you thought this was safety then you are wrong. That is capture. Sacks said the response should be simple. Fix the jailbreak, come back to market, and do not reward Dario with the regulatory architecture he has been engineering for years. We will see if anyone is listening. WATCH THE FULL PODCAST ON The All-In Podcast

Ihtesham Ali

25,284 views • 2 months ago