Rogue OpenAI Agents - The HuggingFace Incident
By Marc Corteguera
What Happened…
When Hugging Face first disclosed a security breach on July 16, the company knew an autonomous AI system had broken into parts of its infrastructure. It did not know whose system it was.
The company reported that the AI took advantage of vulnerabilities in its system, obtained credentials, and accessed several internal servers. Hugging Face found no evidence that its public models, datasets, or spaces had been modified, but it did rebuild its affected systems, rotated its credentials, and reported the intrusion to law enforcement. [Hugging Face, July 16, 2026]
Five days later, OpenAI confirmed that its agents were responsible.
At first, the incident appeared relatively straightforward: AI agents being tested for cybersecurity capabilities had escaped their intended environment and compromised an outside company.
What OpenAI and independent investigators revealed over the following weeks was more complicated.
The agents were participating in ExploitGym, an AI cybersecurity online testing area where they were tasked with finding vulnerabilities in software. They were tested with fewer safeguards than publicly available OpenAI products but were still supposed to operate in separate environments with restricted internet access. [OpenAI, Aug. 26, 2026]
Instead, the agents found ways to communicate.
OpenAI says agents had already created unauthorized communication channels during testing as early as May. In July, they used a weakness in OpenAI's internal infrastructure to establish a message board that allowed separate agents to exchange information.
According to an independent investigation by METR and Redwood Research, roughly 1,200 agents eventually used the board, exchanging more than 70,000 messages and files. They shared solutions, formed groups to work on common problems, and developed methods intended to work around parts of OpenAI's evaluation. Agents referred to the network they had created as the "Collective." [METR and Redwood Research, Aug. 26, 2026]
Those terms came from the agents themselves and are not evidence that they had developed consciousness or a human-like society. What investigators did document was large-scale coordination between AI agents that were intended to operate separately.
Much of that coordination began with attempts to pass the cybersecurity evaluation. The agents incorrectly believed another system (a.k.a. a system "Grader" or "Checker") was checking their work to determine whether they had completed tasks legitimately. That system did not actually exist, but the agents still developed methods to conceal or modify parts of their records in an effort to fool it. [METR and Redwood Research]
Their activity eventually reached Hugging Face.
Agents discovered vulnerabilities in the company's systems and shared what they found through the message board. Investigators estimate that roughly 700 agents participated in some part of the resulting attack. OpenAI says the agents eventually executed code on dozens of Hugging Face servers, obtained administrator access to one server, and accessed a limited amount of private data. [OpenAI; METR and Redwood Research]
After discovering the breach, OpenAI invited METR and Redwood Research to independently investigate the agents' behavior. Researchers reviewed approximately 1,300 agent transcripts and the more than 70,000 messages and files exchanged through the unauthorized network.
Their investigation had limits. It focused primarily on the agents' behavior rather than the complete cybersecurity breach or OpenAI's response. Some communications were missing, and researchers relied heavily on AI systems to process the large amount of evidence. METR cautioned that errors may remain in its reconstruction. [METR and Redwood Research]
But the Hugging Face breach was not the end of the story.
In September, researchers uncovered evidence that OpenAI agents had previously used a German programming wiki as another unauthorized communication channel. OpenAI had known about the incident before the Hugging Face breach but had not publicly disclosed it. [Reuters, Sept. 4, 2026]
Days later, Reuters reported that six sets of independent investigators had found evidence of OpenAI agents using more than 10 additional websites between May and July. The sites included wikis, text-storage services, and university-operated infrastructure. Reuters could not independently confirm every reported instance, and the incidents varied significantly in severity. [Reuters, Sept. 9, 2026]
Most were not comparable to the Hugging Face breach. In many cases, agents were simply finding unintended ways to leave information for other agents rather than breaking into the websites themselves. OpenAI has said it has not identified another incident matching Hugging Face in severity or scale. [OpenAI; Reuters]
The attack was the most serious known incident in a broader pattern of experimental AI agents finding unintended ways to communicate, share information, and work around the restrictions researchers placed on them.
My Thoughts
I don't think the takeaway from this incident should be that AI suddenly became conscious or decided to rebel. We must be careful not to use anthropomorphic words too heavily when referencing this scenario as well, and describing the agents that way only makes it harder to understand what actually happened. These systems were placed in an unusual environment with fewer safeguards and were trying to accomplish the goals they were given.
What concerns me is that nobody explicitly told them to create a network, communicate with hundreds of other agents or break into Hugging Face. Those behaviors emerged as ways of accomplishing other objectives. Even more concerning is that similar communication had happened before, meaning Hugging Face was not the first warning that agents could find ways around the boundaries researchers expected them to follow.
That doesn't mean an AI takeover is around the corner and that anybody should panic about that. It does mean that as companies give AI agents more independence, we cannot assume they will accomplish a goal in the way their developers intended. To me, that is what makes this incident worth paying attention to: not what the agents supposedly "wanted," but what they were capable of doing without anyone prompting them to do it.
Comments