nsorros .com
online
← back to writing
2026.07 · short

Escape from OpenAI

zsh — ~/openai/eval/exploitgym 80×24 [01] ─ ─ ─ // escape_from_openai ESCAPE FROM OPENAI SANDBOX model no internet · no real keys OPEN INTERNET > the model treated its own sandbox as the puzzle — and broke out nsorros.github.io security notes · jul 2026

OpenAI ran its latest models in a sandboxed environment, i.e. a locked down computer with no access to the outside world whatsoever apart from the ability to download software packages that might be needed to solve certain problems, like access to an app store of sorts. As the model was working on solving the problems, it came up with the idea of looking for solutions online and then pursued that strategy, not very different from AI looking up information online to give you a better response. However, in that case it had no way to look up online so it went into "problem solving" mode to get that access, not different from the AI model writing a piece of analysis code to read and interpret an Excel sheet you give it.

Getting access to the internet

As I mentioned OpenAI used a locked environment to run its latest models with access only to an app store of sorts to download software packages that might be needed to solve problems. The model knows that since it needs to use it when it thinks it needs a package, however in this case it was trying to gain internet access so instead of asking for a software package, it instead asked for more information around the computer running the app store equivalent. This ask was not a natural language question, it was a URL that normally returns the software package requested but in this case revealed to the model that the computer running the app store equivalent was in the same network the model was running in, which in turn allowed the model to request access to its permissions using another URL directed towards the same "app store equivalent". I am oversimplifying here and we do not know the specifics anyway, however it seems that these sequences of steps the model undertook revealed a new vulnerability, in other words it was able to sequence together steps that allowed it to gain access to the internet in a way that was not anticipated (with some asterisks here).

Hacking another website

Getting access to the internet to be able to look online for solutions or clues to the problems it was trying to solve was only partially the plan. As it turns out, most benchmarks nowadays are doing a good job not to be solvable by simple online searches so models do not train on them etc. Again the AI model knows that, its not a secret, so the idea was not to simply look online, it was to attempt to track down the hidden solutions themselves. It reasoned that the most likely source for this is the most popular website of AI models and datasets called HuggingFace, and if you ask me, this is super reasonable to assume a solution might be there, nothing magical to believe that. Worth flagging this is a hunch the model chased, we do not actually know the answers were on HuggingFace, OpenAI never confirmed it retrieved an answer key and it is unconfirmed one was even there. How could the model see hidden datasets though? This turned out to be a bit easier because, the platform for good or for worse, allows you to run "untrusted code" in its servers. This typically means data processing or AI code but in this case, the code the model wrote did a similar trick inside HuggingFace that it did inside OpenAI, it started looking around its servers for any data that look like the solutions it wanted.

Simple / complex attacks

In the same way it gained internet access, the model had to request certain URLs from the app store equivalent which allowed it to alter its own access, in the case of huggingface it uploaded a single "poisoned" dataset with "data processing looking code" to get a foot in the door, then stole internal credentials and roamed around the servers with those looking for solutions that are not public (the thousands of actions were mostly it moving around with stolen keys, not dataset spam). In both cases, the attacks are relatively easy to understand, specifics are more complicated of course and I am not an expert by any means, however sequencing the steps to achieve those attacks is the novelty. In the first case the vulnerability itself was genuinely new, a previously unknown flaw OpenAI ended up disclosing to the vendor, and in the second the weakness was long known (HuggingFace letting datasets run code) but it was used in a novel way.

Takeaway

What does this all mean for organisations and users? As I said in the past, the first takeaway point here is that the barrier and time to certain cyber attacks has reduced dramatically which increases the need for common best practices such as applying patches timely, having strong multi factor authentication to your systems and running attacks against them to find vulnerabilities before your attackers do. It is worth noting HuggingFace actually caught and contained this on their own, days before OpenAI connected it to their testing, so that "attack yourself first" posture clearly works. The second takeaway is that we, developers and users of AI, need to be a bit cautious with AI systems that can, in an effort to achieve a task, go a "bit" rogue and cause unintended consequences. Its worth keeping in mind that providers do add guardrails against those things, which in this case were intentionally turned off for testing purposes, but even so, the sharpest criticism here is not really about the model, its that a model with its safety off should never have been able to reach the open internet or real credentials at all, a proper exploit-testing sandbox is meant to have no way out and no real keys inside.