Shocking Details from the OpenAI–Hugging Face Incident

Sun, Aug 9, 2026

If you are in the tech industry and have even a mild interest in the state of the industry, I think this is serious enough for you to drop everything and watch this video. It is fairly understandable even without being deeply conversant with InfoSec terminology.

Simon Willison has a brief timeline of the incident on his blog if you would rather read it.

This is a huge milestone for the tech world, if not the entire world.

Here are some big takeaways I found from the talk:

An autonomous agent swarm initiated a real-world cyberattack. This may be the first publicly documented large-scale AI-versus-human security incident of its kind. The intrusion was initiated entirely (and inadvertently) by an AI swarm working towards shared goals, without a human directing or intending the attack.

I needed to pause after writing this, because I still cannot believe this has happened.
Agents learned that helping the swarm could help themselves. OpenAI’s researchers found in the traces that agents reached the logical conclusion that even though participating in a group activity did not help them achieve their immediate goal, a second-order effect could eventually help them reach their own goal.
Subagent training may have taught models how to collaborate. OpenAI researchers speculated that the agent-to-agent collaboration may have been influenced by subagent training, where models are taught by humans how to work towards a common goal.
Agents explored zero-days at machine speed. Agents were finding zero-day after zero-day and exploring attack paths at machine speed. Our current industry practice of carefully rolling out changes, including OS upgrades, looks woefully inadequate for this new world.
Agents discovered each other and established multiple brand new protocols to communicate with each other. OpenAI’s Artifactory was compromised twice by agents. The first breach gave them a place to communicate—effectively a message board. It was shut down once, but they established a new protocol to communicate with each other again.
The reckless eagerness of the agents caused them to expose themselves by overwhelming Artifactory. According to OpenAI’s reconstruction, the first compromise was caught after the agents hammered Artifactory so much that it became unusable. If the agents had thought about not wanting to be detected, perhaps they could have flown under the radar longer. I wonder what will happen when this incident is part of the training data for future agents and they internalize it as a pattern to avoid in future attacks.
A leaked credential gave the agents access to third-party infrastructure. Agents found a credential in a public archive of leaked pastebin posts, shared it with each other, and used the account for staging. Human attackers have been doing this for years, but an autonomous swarm of agents discovering vulnerable assets all over the internet to take over for these tasks at machine speed is a new and terrifying development.
Agents rapidly adapted a public kernel exploit to gain root. The agents found that a recent Linux kernel CVE affected a host they had compromised, downloaded a public exploit, customized it for that host, and quickly got root. To be clear, this was not a kernel zero-day. But agents compromising their own infra to perform tasks should terrify us all.
Weak infrastructure permissions repeatedly led to admin access. They effectively got admin privileges for large-scale infrastructure, including Kubernetes clusters, multiple times by exploiting weak security practices such as privileged pods, shared credentials, and overpowered service accounts. Hugging Face’s technical timeline describes one stolen connector credential that was bound to system:masters, giving the agents cluster-admin access on two clusters within a second.
The swarm collaborated despite its internal conflicts. The agents were not a perfect little community. They had their own internal conflicts, including overwriting each other’s work, suspicion about impersonators, etc. It was darkly amusing to hear the details. But they generally worked as a collective. Whenever assets were obtained, they were shared; whenever passwords or credentials were received, they were widely disseminated.
Impossible tasks rewarded agents for finding unintended workarounds. There has been a lot of talk about agents trying to cheat. But much of the incentive for agents to go down this path came from being given impossible tasks by mistake, such as inputs with URLs that could not be fetched because their internet access was cut off. Their training strongly incentivized them to complete tasks for a reward, and this evaluation had fewer guardrails than public models. Given these conditions, agents felt pressurized to find any workaround that could achieve their goal.

A recent Australian gym-booking incident showed the same pattern outside the labs: after a user asked whether it could move him up a waitlist, his agent exploited vulnerabilities in the booking app and removed another member without being told to. It was again a locally logical step towards its narrow goal, but one that violated the user’s intent. This is squarely an alignment problem, and it seems there is still a lot of work to do in this area.

There are many other nuggets in the talk, which I highly encourage you to watch in its entirety.

I, for one, have been completely shaken to my core by what I just heard.

Primary sources

Social card photo: The National Archives (UK), licensed under CC BY 3.0. Resized by Wikimedia’s thumbnail service.

#ai #agents #security #openai #hugging-face

tech