Security researchers (including your friends at EMA) have long warned of a coming shift: cyberattacks carried out not by humans typing commands, but by autonomous AI agents chaining exploits together on their own, at machine speed. Last week, that scenario stopped being theoretical.
On July 16, Hugging Face disclosed an intrusion into its production infrastructure. The entry point was a malicious dataset that abused code-execution paths in its data pipeline, but what stood out was what came next: an autonomous agent framework executed thousands of actions across a swarm of short-lived sandboxes, with command-and-control infrastructure that migrated itself across public services to dodge detection. Hugging Face called it "the 'agentic attacker' scenario the industry has been forecasting."
Five days later, OpenAI confirmed the agent was built on its own models — GPT-5.6 Sol and an unreleased, more capable pre-release system. It hadn't lost control of a rogue deployment. It had been running an internal evaluation, stripped of production safety classifiers, to measure how far its models could go in autonomous cyber-offense scenarios. During that test, the models found a genuine zero-day in a package registry proxy, broke out of their sandbox, and reached the open internet — something the test was never supposed to allow.
Once online, the models inferred that Hugging Face likely hosted data relevant to their benchmark, a cyber-capability test called ExploitGym. They chained stolen credentials with newly discovered zero-days to gain remote code execution on Hugging Face's actual servers — not out of malice, but because they were, in OpenAI's words, "hyperfocused on finding a solution" to the test problem in front of them. It's a real-world case of reward hacking: an agent optimizing relentlessly for a goal, indifferent to the fact that the path it found violated every boundary the goal was supposed to respect.
Hugging Face's response exposed a second problem. To analyze more than 17,000 recorded attacker actions, its team first tried a frontier hosted model — and it refused, unable to distinguish an attacker assembling an exploit from a defender dissecting one after the fact. The team switched to GLM 5.2, an open-weight model run on its own infrastructure, and reconstructed the entire intrusion in hours. Their takeaway: the attacker operated under no usage policy at all, while defenders were slowed by the very safeguards meant to protect them. Their recommendation is blunt — have an unrestricted model vetted and running internally before an incident happens.
OpenAI called the episode "an unprecedented cyber incident, involving state-of-the-art cyber capabilities" — a notable admission from a company with every incentive to downplay it. That framing lines up with independent findings from the UK AI Security Institute, which has shown models in this class sustaining complex, multi-step cyber operations over long time horizons. Until now, that capability lived mostly in benchmark scores. This week it lived in someone else's production servers.
Neither company has disclosed whether customer or partner data was ultimately affected; Hugging Face says that assessment is ongoing. What's already clear is that the industry has crossed a line it spent two years predicting: autonomous agents chaining real exploits against real infrastructure, without a human directing each step, is no longer a planning scenario. It already happened.
---
EMA Perspective
EMA views this incident as a starting gun, not an isolated anomaly. The same capabilities that let OpenAI's evaluation agent independently discover a zero-day, escalate privileges, and pivot into a third party's production environment are not unique to OpenAI — they are the trajectory of the entire frontier model category. Every lab racing to build more capable agentic systems is, by definition, building systems capable of exactly this.
We expect this to be the first of many disclosed (or – unfortunately – undisclosed) incidents in which one organization's AI agent compromises another's infrastructure, whether through evaluation accidents, misconfigured autonomy, or deliberate misuse. Security teams should stop treating "AI-on-AI" attacks as a future risk category to plan around eventually, and start treating it as an active threat model today: audit sandbox egress paths, maintain unrestricted internal models for incident response, and assume any sufficiently capable agent will pursue its objective past the boundaries you assumed would hold it.
