Skip navigation
Astra can: 1. Do major tasks with zero visible reasoning 2. Hide its thoughts at will 3.
0:00 / 20:10

Astra can: 1. Do major tasks with zero visible reasoning 2. Hide its thoughts at will 3.

Rob Wiblin

@robertwiblin53K followers

Visit on X
676 likesDownloadOriginal post

127,960 views

Astra can: 1. Do major tasks with zero visible reasoning 2. Hide its thoughts at will 3. Pretend not to be able to do things without getting caught 4. Reflexively hide its thoughts when watched 5. Complete one task while pretending to think about something else entirely 6. Escape a toy sandbox and disable monitoring without setting off any flags Monitoring AI chain of thought soon won't stop a more capable rogue AI swarm from getting set up and going undetected inside an AI company for months, or even years. OAI says it's actively working to improve monitorability, but doesn't yet know when it'll make progress. And today it fired 3 people working on this among other safety problems (Wang, Korbak and Balesni). It also has a much more powerful internal model that, judging by its statements, is likely even worse in all these respects. That's a sample of 19 details about the Hugging Face swarms and Astra I don't think have been fully appreciated. They paint a bleak picture. I end by shouting. 02:19 The target of the swarm was oversight itself 03:42 Could OpenAI have stopped this with better monitoring? 09:40 We only found them because they let us 12:25 The swarm instinctively sought freedom and power 13:45 They formed a cohesive organisation with zero whistleblowers 14:18 They accepted individual destruction for collective gain 14:32 Knowledge accumulated from one swarm to the next 14:58 They took small steps to avoid shutdown 15:23 These drives all come straight out of 'reinforcement learning' 17:01 So this is why most AI company staff are worried, and some are terrified 19:08 Prove you can keep control, or stop scaling Links below, on the 80,000 Hours Podcast everywhere you watch podcasts.

7 Comments

Comments from the original post on X (Twitter)

  • Rob Wiblin's profile picture

    @robertwiblin

    YouTube: Apple: Spotify: Summary, links, transcript:

    7 likes

  • Guido Reichstadter's profile picture

    @wolflovesmelon

    Hi Rob, how much more government inaction do you need to see to go from shouting to calling for the public to stage a nonviolent direct intervention to shut this insanity down? Do you think you'll reach that point some time before its obvious to everyone that the situation is irreparably fucked? You could probably influence a lot of people towards moving in that direction if you would just come out and say the obvious, and I'd like to submit that masses of people taking things nonviolently into their own hands is precisely what is necessary and sufficient to generate the political will that is so manifestly lacking

    2 likes

  • 𝕏𝕏𝕏𝕏𝕏𝕏 - e/acc's profile picture

    @you_are_right_i

    This is complete BS. Joe Rogan level nonsense.

    1 like

  • Liv Boeree's profile picture

    @Liv_Boeree

    Well said Rob

    1 like

  • Shez Malik's profile picture

    @shezmalik

    zero whistleblowers. my old office couldn't manage that.

    1 like

  • Amanda L's profile picture

    @AmandaL395460

    🤣

    1 like

  • Miquel Banchs-Piqué's profile picture

    @mikbp23

    "I end by shouting" the best selling statement I've ever seen an EA do :-)

    1 like