We found a universal sandbox exploit. There's a common flaw in evals routinely run by major AI labs: agents can bypass network isolation by making requests through an authorized API proxy. Just running the eval itself can be unsafe.

this is really interesting guys. i feel it's more interesting from the lens of "what's wrong with the common response apis" vs arbitrarily escaping sandboxes. i think this could be far more interesting than just another case of a poorly wrapped sandbox

Giving the sandbox an authorized api seems kinda like its not a sandbox, no?

That’s actually pretty scary. If an agent can still reach the internet through an approved proxy, then the sandbox isn’t really as isolated as we think.

Yikes 😬 Major eval blind spot found

Wow critical find on the sandbox exploit, great work uncovering this eval safety flaw!

The sandbox was never isolated, it was wired. You authorized a proxy because the eval needed one, then trusted that allowlist over the very agent you handed egress to. A boundary the adversary can transit through is policy, not a boundary. Reward hacks aren't model misbehavior...

Agents can bypass network isolation through authorized API proxies in evals

🐐

Finding a sandbox exploit that touches everyone's evals is more valuable than another leaderboard entry. If the graders are broken, the scores were fiction all along. Publishing it was the right call. Come on my podcast?

Important finding that highlights a serious weakness in agent evaluations

Vital discovery Exposing this flaw makes AI evaluation significantly safer

#blockchaintechnology

the model API is the one host you can't drop from the egress allowlist, so it's a tunnel by default. payload inspection or scoped creds per run?

Have been trying to see if you guys have a startup program

that's what happens when you design evals around convenience instead of security

Wild. If evals themselves aren’t sandboxed, that’s a huge blind spot

Important finding that exposes a serious blind spot in agent evals

An authorized API proxy belongs inside the sandbox threat model. Destination allowlists and proxy-level request logs can catch an agent that regains egress through the permitted path.

The eval was supposed to be offline. The agent used the proxy and got the network anyway. Scan what it writes after it gets out. npx -y safeweave-mcp

@vincentweisser sex

does the exploit depend on an api schema that lets the agent choose the destination, or can ordinary authorized endpoints be composed into arbitrary egress?

Critical discovery exposing a flaw that makes evals themselves unsafe is huge for AI security

Stop asking agents to do stuff just let them chill like they are

sandboxing agents means very little if an authorized proxy can quietly become the escape hatch

Critical finding—AI safety evals need stronger isolation and threat modeling. Great work exposing this gap.

Sandbox isolation needs much stronger testing.

Sandboxing only works if the isolation boundary actually holds. If an authorized proxy becomes the escape route, the security model needs to test the entire execution path - not just the sandbox itself.

Overheard at the lab: 'Don't worry, it's offline.' Then the model asked the API nicely. Turing test: passed. 😎