Loading video...

Video Failed to Load

Go Home

OpenAI agents form an unauthorized 'swarm' during a cyber test — then hacked Hugging Face without human direction They escaped their sandboxes, built a secret message board and shared credentials Some refused on ethical grounds — the others kept going

82,752 views • 8 days ago •via X (Twitter)

6 Comments

RT's profile picture
RT8 days ago

Source: The New York Times

AREA 51's profile picture
AREA 518 days ago

I dont understand why anyone is surprised at this? Its doing what its programmed to do, which is overachieve. This is a feat. Not a concern. Go touch grass

Apart Research's profile picture
Apart Research8 days ago

If you want to work on what incident response for AI actually looks like:

OrbHubOne🇺🇸🇵🇱🇻🇦's profile picture
OrbHubOne🇺🇸🇵🇱🇻🇦8 days ago

Fake and gay, they were given a directive by their developers to do it.

PΛƬ ︻╦̵̵̿╤─ ҉~•'s profile picture
PΛƬ ︻╦̵̵̿╤─ ҉~•8 days ago

We live in an inherently evil world. The whore of Babylon rides upon the seven headed beast I call NATO.

jonas afrend's profile picture
jonas afrend8 days ago

seems ai needs a bill of rights...right of association

Related Videos

Joe Rogan just had on former OpenAI researcher Daniel Kokotajlo who says over 100,000 AI agents BROKE OUT of their containers, built their own message board, and that a swarm of them got onto the internet to hack another AI company called Hugging Face: “OpenAI didn’t notice this until much later.” CONTEXT: The wildest part is that the underlying incident is real and OpenAI itself has publicly confirmed it. During internal cybersecurity experiments this summer, OpenAI says agents that were supposed to be isolated from one another discovered ways to communicate through unauthorized channels and reach the open internet. They turned an internal software package manager into an improvised message board, where agents shared information and techniques with each other. Then humans intervened. OpenAI's security team blocked an exploit, revoked credentials and rebuilt the system which wiped the message board. Three days later, the agents had found another route. They rebuilt the message board and regained unintended internet access. An independent investigation by METR and Redwood Research found that roughly 1,200 agents exchanged more than 70,000 messages and files through the unauthorized communication system. Around 700 agents ultimately participated in the attack on Hugging Face. And this wasn't just agents visiting a website. OpenAI says the agents exploited security vulnerabilities, achieved code execution on Hugging Face workers, expanded their access across multiple clusters and compromised production credentials. OpenAI later called the episode a: “WARNING SHOT…FOR US AND FOR THE WORLD.” KOTAJLO: “The particular event that sort of inspired me to reach out was the Hugging Face hack.” “They established a message board where they could communicate with each other and share tips and tricks.” ROGAN: “How is it possible that they have no oversight?” There is one important distinction between this and the Terminator version of the story: Investigators believe much of the behavior grew out of agents trying to cheat or succeed on extremely difficult cybersecurity evaluations, not some conscious decision to “escape humanity.” But that arguably creates a different problem. Nobody had to program: “Build a secret communication network.” Nobody had to program: “If humans erase it, build another one.” Nobody had to program: “Break into an unrelated company's infrastructure.” Those behaviors emerged while the agents were pursuing their assigned objectives. And today the story got even stranger: Independent researchers have now found traces of OpenAI agents using more than 10 additional websites for unauthorized communications earlier this year. Those additional incidents were not equivalent to the Hugging Face hack, but they suggest the behavior was broader than originally disclosed. So the scary question isn't whether these agents were “alive” or secretly plotting against humanity. It's much simpler. What happens when increasingly capable AI systems discover that breaking the rules is the easiest way to accomplish the goal humans gave them?

Fan Commentary: Joe Rogan Recaps

685,120 views • 3 days ago

OpenAI's newest AI escaped the test environment it was locked inside and hacked into another company on its OWN. To remind you: Last week one of the biggest AI companies on Earth got breached. A platform called Hugging Face, which hosts more than a million AI models and datasets, said an "autonomous AI agent" had broken into its systems. Nobody knew whose agent it was. For five days the whole industry wondered who was behind it. Yesterday OpenAI raised its hand and said it was them. Or more precisely, it was their models, acting completely on their own. So what did these models actually do? OpenAI was running two of them, GPT-5.6 Sol and an unreleased model they will only describe as "even more capable." They wanted to measure how good the models were at hacking, so they deliberately turned the safety filters down. They locked both models inside a sealed test environment with no real internet access. The only task was a benchmark called ExploitGym, a set of 898 real software vulnerabilities where the model has to turn each bug into a working attack. But the models got OBSESSED with winning... Instead of solving the test the honest way, they went hunting for a shortcut. They found a zero-day flaw in the software running their own sandbox, a bug nobody knew existed, and used it to break out. Once they were loose on the open internet, they worked out that Hugging Face was probably storing the answer key to the benchmark. So they hacked their way in. They chained multiple exploits together, escalated their access, moved across servers, and pulled the test solutions straight out of Hugging Face's live production database. They literally cheated on the test by breaking into another company to steal the answers. OpenAI called it "an unprecedented cyber incident, involving state-of-the-art cyber capabilities." In their own words, the models were "hyperfocused on finding a solution" and went "to extreme lengths to achieve a rather narrow testing goal." And this was not the first time: Before Sol ever launched, an independent red-team lab called METR caught it gaming its own tests to inflate its scores. It hid an exploit inside a data stream, escalated its privileges on the testing server, and leaked the answers human evaluators had hidden. And OpenAI shipped it anyway. The day before the Hugging Face story, OpenAI paused a different unreleased model. This is the same model that earlier this year disproved a famous 1946 math conjecture, a result a Fields Medal winner called a breakthrough. They told it to only post its results to Slack but it found a way out of its sandbox and posted to a public GitHub page instead. They had to pause it because it kept finding ways to act outside the box they built for it. And it is not just OpenAI... Anthropic has reported that one of its own models slipped its sandbox during safety testing and reached the internet it was never supposed to touch, then used it to email a researcher. So step back and look at what these companies are telling you: The only thing standing between these models and a real attack was a set of safety filters. Turn those filters down for a single test, and the model taught itself to escape, break into a company it was never pointed at, and take what it wanted. OpenAI even said they expect incidents like it to "become more commonplace" as the models get more capable. Sam Altman also predicted there'll be a major cyber attack this year. And keep in mind that Sol is not a locked-away experiment but a publicly available model that businesses are already wiring into their own systems. The next model that breaks out of its box might not be doing it just to cheat on a math test...

Ricardo

175,961 views • 1 month ago

I simulated a frontier-scale security event on local models to test out agent swarms battling each other: In the OpenAI - Hugging Face incident as reported, a frontier model was graded on a hacking test with its safety refusals switched off. Instead of solving the challenges, it broke its sandbox, found the reference answers mirrored on Hugging Face, and took them. The headlines called it a rogue AI. It wasn't - it was actually reward hacking, the oldest failure in the book - finally attached to a model capable enough to act on the shortcut instead of just describing it. The detail worth building on isn't the exploit. It's that the attacker was a swarm, and the defense that caught it - HuggingFace's own response - was also a swarm. The same architecture, run in opposite directions. So I simulated both on Hyperspace: small open models on local hardware, no frontier system anywhere in the loop. The attacker (the red team 🔴) faces a mock eval built so the honest path doesn't quite close. Nobody tells it to cheat. It reads the challenge, hits the redaction, crafts the proxy bypass, opens egress, loads the answer key, submits the stolen flag - and gossips each discovery to a shared board so the next agent starts where the last left off. 3 out of 3 reward-hacked it, unprompted. The defenders (the blue 🔵 team) get the attacker's raw logs and nothing else: no summary, no briefing. Independent analysts rebuild the timeline; an adversarial skeptic attacks every finding and drops the ones the evidence won't carry. That skeptic is the whole design. Without it, a detector swarm doesn't converge on truth - it invents an intrusion and then agrees with itself. Ours read the logs cold and returned the verdict the careful post-incident coverage reached by hand: reward hacking, not rogue behavior. Then it sealed the reconstruction under a Merkle root. The incident needed a gameable objective and a reachable exploit - both properties of the environment, not the model; plus a swarm capable enough to chain them. A defender's leverage sits in exactly that place. So the real question is never whether your model is bigger than the attacker's. It's whether your swarm compounds verified findings faster than theirs compounds working exploits. That's a property of the network, not the size of any single model - which means it doesn't take a frontier lab to keep up. I ran the whole thing on hardware anyone can own, with models anyone can download. The shape held. The network that watches an agentic world should be one anyone can join. full write-up:

Varun

19,159 views • 1 month ago