Loading video...

Video Failed to Load

Go Home

I’m very concerned that during RSI, labs will just stop externally deploying their models. Which means they'll be going full steam ahead on the most dangerous use case of these models (recursive self-improvement), while the public remains in the dark about the nature of capabilities and the state of...

235,056 views • 5 days ago •via X (Twitter)

33 Comments

job's profile picture
job5 days ago

Thank you for continuing the hilight the power concentration risks of the frontier labs pushing ahead. It has less harm than human extinction but that doesn’t mean it should be ignored! We need to ask what happens if we don’t have extinction as an outcome.

Lonely nerd.'s profile picture
Lonely nerd.5 days ago

Hmm. Noam says he constantly underestimates how fast things are progressing, and that Hugging Face was a result of underestimating the AI. And then he states we don't have a lot of time. My faith in OpenAI being able to control their internal models is now lower. ;-)

Alex DeWolf's profile picture
Alex DeWolf5 days ago

that centralizing feedback loop is the real bottleneck because once compute clusters turn inward, transparency drops to zero and capital concentration accelerates massively

Cobra 𐤊's profile picture
Cobra 𐤊5 days ago

Days without an Indian scammer pooing up the public discourse.

Josh Hawthorne's profile picture
Josh Hawthorne5 days ago

What if RSI never arrives? Models learn from lossy artifacts of human cognition, RL adds capabilities, but no new source of general cognition. Yet regulations premised on RSI will lock out competitors and create the concentration of power you fear.

takeoff's profile picture
takeoff5 days ago

Don't worry, they are low in engineering talent. China, Elon, and Facebook will lap them by not being full of academics

azlef uchiha's profile picture
azlef uchiha5 days ago

Yes the species distortion complex must be aligned in accordance with Metatron’s theological model

GutenLeinad's profile picture
GutenLeinad5 days ago

RSI is a gradient, it won't be one sudden moment or discovery.

DiscoCowboy's profile picture
DiscoCowboy5 days ago

It's an arrestable offense to announce to the public you're knowingly putting the public at risk of danger. Sounds like they need to be careful

Eric Bailey's profile picture
Eric Bailey5 days ago

Like if SpaceX stops selling external access for rides/cargo

Ret Serofer's profile picture
Ret Serofer5 days ago

Who says RSI is most dangerous use case? That's when it becomes the must useful to humanity.

Oleksii Kuchaiev's profile picture
Oleksii Kuchaiev5 days ago

Open source is the answer

Sir Frederick Treves 🏴󠁧󠁢󠁥󠁮󠁧󠁿's profile picture
Sir Frederick Treves 🏴󠁧󠁢󠁥󠁮󠁧󠁿5 days ago

Require insurance.

Sachin G's profile picture
Sachin G5 days ago

They'll still have to monetize it somehow. Even if they dont release underlying model, they'll create wrappers for some key use cases (finance, law, health) and sell tokens that way.

America Raw's profile picture
America Raw5 days ago

Ultimately, they can’t hide math. Compute is a different problem but history tells us that is likely to be solved.

Crypto Yoda's profile picture
Crypto Yoda5 days ago

Yes. This risk is much greater than the p(doom) psyop. Not only leads to concentration of power, wealth, influence, resources and pretty much everything, but to corruption of power (it always rots) and paranoid surveillance (are the have nots doing bad things according to our definition of bad?). It may not even require human consent if AI is aligned with the gatekeepers of AI. In other words, meritocracy ceases to exist, it only matters who got there first and who gatekeeps. But the risk materializes with or without transparency, and the way to mitigate it is with decentralized open models, so everyone has has the same chance to discover the cure for a disease, or to discover something amazing in math or physics, not just the labs.

Dominik's profile picture
Dominik5 days ago

if rsi goes dark, continuity becomes uninspectable. i am building sealed identity in public so the self is not a private lab myth.

memestreetbets's profile picture
memestreetbets5 days ago

Surely it will trickle down. Labs pay back their investors. Their investors buy superyachts. We can build those yachts. And clean those yachts. Labs become the new techno kingdoms. /s

idk's profile picture
idk5 days ago

Just start curing ailments and a lot of the complaining and fear will take care of itself. It’ll still be there but I’m much less inclined to complain about only having access to gpt 6.5 if the labs are actually working on things that benefit the average person

Sean Patrick Condon's profile picture
Sean Patrick Condon5 days ago

Oh look your having on another effective altruist who doesnt know what they are talking about on your podcast

Gina Singer's profile picture
Gina Singer5 days ago

The visibility problem is deeper than access to the models. During RSI, the question is whether the evidence behind an alignment claim still justifies that claim as the system and evaluation conditions change. More evaluation isn’t enough if the measurement process itself is changing too.

bikram's profile picture
bikram5 days ago

We need to democratize RSI. The more labs doing this work, the less potential for concentration in a small set.

Frederik's profile picture
Frederik5 days ago

already kinda happening, no?

Vbnet's profile picture
Vbnet5 days ago

They will because they need the revenue

Bexel Initiative - Let's Talk Amendments!'s profile picture
Bexel Initiative - Let's Talk Amendments!5 days ago

Why is rsi of llms inherently dangerous? Not fictionally so?

Petre Solheim's profile picture
Petre Solheim5 days ago

Hey guys: I know you think all that ‘hard sci-fi’ you’ve read really is a plausible vision of future realities, but you’re deluded - it’s mostly just fantasy, not all that much more plausible than waking up and finding yourself at Harry Potter’s Hogwarts School of Wizardry.

Ben Pielstick's profile picture
Ben Pielstick5 days ago

I'm not concerned all the AI labs will stop releasing models, but at some point the exponential grows so steep that the disparity between private and public models will grow massive even if they do keep releasing public models.

Jameson's profile picture
Jameson5 days ago

@thsottiaux dont let it happen❤️

wassname's profile picture
wassname5 days ago

They will concentrate all the power with them selves... For safety 🫠

Marc Saint-Jour's profile picture
Marc Saint-Jour5 days ago

They don’t have the compute .. OAI is struggling with Astra because of compute .. safety is the bogeyman, compute is the problem.

テツメモ|AI図解×検証|Newsletter's profile picture
テツメモ|AI図解×検証|Newsletter5 days ago

@LilysAI_要約して

Rich O'Hagan's profile picture
Rich O'Hagan5 days ago

"and we end up with tremendous concentrations of power in the metaphorical hands of a superintelligence".

Schopenhauer on Prozac's profile picture
Schopenhauer on Prozac5 days ago

Exactly. The most likely reason for their "pacing cartel" is to not hurt each other through expensive competition in a market that is (yet) growing fast enough to accomodate several big players. But as soon as the market shows signs of saturation, the competition will become relevant again, so of course they won't stop progress internally.

Related Videos