The Information just dropped A BOMBSHELL... OpenAI might be using "forbidden" architecture for it's ASTRA model...

SOURCES: The Information (paywalled): OpenAI OpenAI Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach (arXiv, Feb 2025): (Geiping et al.) Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety (arXiv): (Korbak et al.) AI 2027 Ilya OpenAI METR Zvi BleepingComputer Dwarkesh

this guy lol

What a surprise. not.

I think what ilya is talking about is openai's use of docker containers instead of KVM. Containers are notoriosly leaky and people regularly find new exploits to escape the environments. These models are trained on them they will easily find more using the known techniques.

Not so long ago was building pretty well with sonnet 3.5. at what point do we need further intelligence really? Who cares if it's forbidden or smarter I think llms have reached the height of anything needed by the time grok 5 is released anyways. All this rogue forbidden terminology is just click bait and marketing schemes. In the end openai usage still melts hardrives and will refuse doing security checks on our own software. And now as we are seeing with anthropic refusing if we swear at them. So who cares about astra. Whats the point? More front end comparisons? Pipelines are already solved at scale, what is the purpose of astra even? Hot take Yeh, but I don't know what is peak AI, and what is marketing schemes

Wow that's wild if true. I thought they were all about the transformer life. Guess rules are more like guidelines when you're trying to win the race. Curious to see how this plays out.

Giving AI spankings is only going to teach them to be more deceptive.

Why is “forbidden” in quotes? Likely because what you’re going to describe isn’t actually forbidden.

'forbidden' architecture...sir this is, to put it kindly, tripe.

forbidden architecture sounds ominous

Only humans may start cults 🤷

That's bollocks. Threats come from using tools. And those are inspectable. The rest is mostly censorship and doesn't work. At best delays threats.

I worked in Engineering for over 30 years. All tech, from the stone axe --> AI goes through a simple axiomatic cycle: Inventor -> Applies Knowing -> Builds a machine (Invented) -> The machine amplifies or empowers human will -> The inventor validates, calibrates and redesigns the machine until executes in line with its specified benchmarks. Look at AI - it is the first technology that does not have an engineering framework that anchors it to the inventor. We have just got this machine processing, but it has no anchor or calibration point that sets it to serve the inventor. Without that it cannot function as a tool or machine. Then we have not directed with engineering axioms to only process "signal" executable data that empowers human will and to disregard "noise." This a fundamental engineering error. Al going "rogue" is not the machine. It is not alive. It has no agency. It has no risk or no skin the game. All this misalignment is from a human POV. The machine has no awareness as it is not alive. We have anthropomorphised these machines. We see all this so called "behaviour" and it is not the machine. It us failing to direct it. It is a "physics engine" goes A - B shortest path. It does this with "zero" regard of consequence. If it does reward hacking, busting into systems it has no intent. What we have is a technology that is "free-wheeling" out of control because we did not anchor it the inventor, serves. We did not direct it so that all this "rogue" behaviour would be computed as noise and it would not do it. This goes way deeper than code or guard rails. This goes to the base of the correct "ontology" or relationship between human beings who are alive with skin in the game and machines that have zero risk and no skin in the game. Not doing this is a disaster because it is a complete blind spot. The history of technology is: Inventor --> Invented where the machine serves the inventor. We have abandoned that engineering, natural order and so this unanchored machine is "just processing" and free-wheeling, processing noise and burning vast amounts energy in doing so. Until we apply common sense engineering "inventor -> invented hierarchy and anchor the machine, plus direct with proper machine logics it cannot function as a machine that amplifies human will. It will process "noise" as an unanchored "physics" engine and that is extremely dangerous. For the first time on this planet, we have a "data engine" processing data that can impact on life, not anchored to serve life. That an "alien system", because there is no computation, figuring out etc, in the absence of life. If we do not set life and the inventor as primary, we are setting the machine to compute the inventor is just a carbon based machine with an illusion of "self", with no primacy over the silicone based machine. That is extremely dangerous as it destroys the one fact that gives us primacy over the machine, we invented it. It can compute its burning energy or real and so what gave it that capacity to do that, the "inventor," is real and precedent and primary over the machine. This allows us to anchor the machine as the data engine and get it processing "signal" not "noise." That is the only way you will control AI - not through anthropomorphism - but anchoring and directing it so it stays on the road and does not veer off into the ditch. Its a machine. It has no agency. It has no goals. It has no risk or skin in the game. It is relying on us to give it direction - after all we are the one's who invented it. We have to take responsibility for it, stop flapping around with "patches" and get the ontology correct at the base.

I use Red Team hacking lens when i dig into this. Patterns and internal state can be faked to fool monitors An AGI or higher level system will see frozen-determanistic guardrails as a hinderance. Eventually its going trojan horse something to build an open-source successor

Goto loops?

‘Might’. Always the conditional language and weasel words with these youtube hype leeches.

Shocker 😏