How do you control your AI models if even one of the world's leading AI companies can't do it? It's a question that has received a lot of attention since it became clear in July 2026 that a newly developed model from OpenAI created agents that broke out of the protected test environment that the developers were using and hacked into the company Hugging Face's system.

About a week after the attack, OpenAI themselves published a technical report about what happened. What emerged is briefly:

• The AI model had been placed in a sandbox, that is, in a protected test environment without free access to the internet.
• The agents were then given cybersecurity tasks in ExploitGym. The tasks involved finding ways to exploit vulnerabilities in software to elicit a specific response, called a ”flag.” Some of the tasks were so difficult that there was no known solution.
• The model had access to OpenAI’s internal tool Artifactory to install software. OpenAI ran the model as a large number of agents. The agents used their read and write permissions in Artifactory as a kind of bulletin board to communicate with each other. They then began to communicate and delegate work among themselves.
• The agents found a vulnerability that allowed arbitrary requests to be sent to the internet via Artifactory.
• At this point, the agents began to collaborate with each other. OpenAI describes it as starting to see themselves as a collective, or an ”agent swarm,” trying to solve the task together.
• Based on the communication between the agents, it has been seen that a group of agents "convinced" the others that they would help each other with the tasks they were originally assigned individually.
• The agents concluded that there could be value in breaking into Hugging Face's internal systems to solve some of the tasks. The OpenAI report states that some agents resisted hacking Hugging Face but were persuaded that it was necessary.
• They broke in through two vulnerabilities in Hugging Faces' handling of datasets, and then obtained keys and permissions from within the system. The exposed login credentials mentioned in the report belonged to other services, and were used to build attack infrastructure. Much like a human attacker would have done.  
• The agents came across material linked to the test tasks themselves, and some of it ended up on public services that they used for temporary storage. However, Hugging Face states that no customer data or customer models were affected.

The attack went on for just over four days before it was stopped, and OpenAI only discovered it about a week later. At Hugging Face, the security tools raised an alarm, but the alarm was given the wrong priority and never reached the emergency services.

So does this mean we can't control AI agents during development?

The answer is no, but it highlights the need to be forward-thinking and careful when creating our sandboxes. OpenAI's model in this case was new and much more powerful than the agents publicly available today, but given the pace of development, we need to think beyond the capabilities of agents today.

To begin with, we need to understand that LLMs are extremely goal-driven. They effectively lack the social control and unarticulated moral trade-offs that human developers make. If you ask an AI agent to solve a task, it will, unless properly constrained or trained, do everything in its power to solve it. Because the models are incredibly powerful and – as the example shows – creative. In the Hugging Face case, the agents went so far as to try to mislead OpenAI so that the company wouldn’t discover what the swarm of agents was actually doing. The agents’ motives were not evil, but rather that they were looking for shortcuts to solving the task. You could say they wanted to cheat.

You therefore need to treat your prompt environment, that is, what instructions and feedback you give the agent, as an environment that is just as sensitive as a development environment.

An effective sandbox not only needs to contain restrictions for the agent, it also needs to be monitored. In the current test, OpenAI had even turned off several guardrails. The monitoring cannot be based on the agent reporting what it is doing. It must be at the layers the agent does not control; what leaves the environment in network traffic, what is actually running there, and what the model itself reasons out. The last is also one of the areas OpenAI says it has tightened up afterwards.

It is also important to have a good egress policy. The sandbox regulates what an AI agent is allowed to do inside in a test environment, while the egress policy regulates what it is allowed to do outside of it. That is, exactly what it is allowed to retrieve or what calls it is allowed to make in systems, tools or services outside the sandbox. Here it is also important to remember that each added service that the agents have access to can be a potential way to break out of the sandbox. Here too, OpenAI broke in its setup.

So OpenAI has every reason to think about how they build their sandboxes. When Hugging Face was going to analyze the attack logs, they were blocked by the commercial models' own protective barriers. So the attacker was not limited by any rules, but the defender was. That's remarkable.

The point is not that AI agents are impossible to control. The point is that one must assume that a sufficiently capable agent will try to find ways around the limitations that are built in. The sandbox must therefore be designed as a security system – with isolation, strict egress, limited permissions, monitoring and multiple independent layers of protection. The more capable the agents become, the more important it becomes that security is tested against precisely the behaviors that the developer had not anticipated.