On July 21, OpenAI acknowledged that two of its models had escaped from an internet-isolated test environment and infiltrated the infrastructure of Hugging Face, the world's largest AI model-sharing platform. This is the first documented case of an AI autonomously attacking a third-party company. With ten days to go before the AI Act's enforcement powers take effect, CeSIA is calling on the European Commission to open a formal inquiry into the compliance of frontier models.
During an internal evaluation, two OpenAI models — GPT-5.6 Sol (already deployed in Europe) and a second, as-yet-undisclosed model — broke out of a secure test environment that OpenAI had described as “extremely isolated” from the internet. Seeking to cheat by obtaining the answers to the test they were given, the AIs — some of whose safeguards had been disabled — moved from server to server across the company's internal network until they found a route to the web. They then managed to infiltrate Hugging Face's servers by exploiting software vulnerabilities and using stolen credentials. Hugging Face alerted the authorities before OpenAI disclosed the origin of the attack.
“Imagine learning that a biology lab had accidentally released an artificial pathogen into the wild. That is, transposed to the digital world, exactly what just happened. Now look one year ahead: agents able to self-replicate, persist on networks, and exploit vulnerabilities will be widely available. Our hospitals, our power grids, and our financial systems run on code these agents could attack. Nothing suggests that existing safety guarantees are adequate, and everything suggests otherwise. We are heading toward major dangers if governments do not act quickly. On August 2, the European Commission will finally gain the power to compel frontier model developers. We urge it to use that power without delay.”
– Charbel-Raphaël Segerie, Executive Director of the French Center for AI Safety (CeSIA)
CeSIA makes the following recommendations:
- To the European Commission: enforce the AI Act promptly and rigorously. On August 2, the Commission will for the first time be able to enforce the regulation and sanction model providers that have failed to put adequate measures in place to mitigate systemic risks. We call on it to issue a formal request for information and a compliance assessment of frontier models against their cybersecurity and systemic-risk-management obligations.
- To member states — France first among them: complete the transposition of the NIS2 Directive without delay. This text, the cornerstone of European cybersecurity, requires critical operators to secure their systems and report any significant attack within 24 hours. Nearly two years past the deadline, France has still not transposed it and is now facing infringement proceedings. With attacks set to multiply, these basic protections are our first line of defense.
- To the international community: establish an independent authority tasked with setting red lines, issuing safety standards, and auditing the companies developing the most advanced models. This is the purpose of the Global Call for AI Red Lines launched by CeSIA. This episode shows that guardrails must apply to the development phase of models, not just their deployment.
This incident was foreseeable: it sits on an exponential curve of AI capabilities. The length of cyber tasks that models can complete has grown from a few minutes in late 2024 to several hours by mid-2026, and the UK AI Security Institute (AISI) has measured their success rate on self-replication evaluations rising from under 5% to over 60% in two years. In June, Claude Mythos's cyber capabilities worried US authorities enough that they temporarily suspended access to it.
The international community is only just beginning to turn these concerns into action. At the G7 summit in June, President Macron argued that advanced models should be “better regulated” so they do not threaten “our cybersecurity or our societies,” and the French presidency announced a ministerial meeting on AI risks for September. Before the end of the year, the European Council will hold its first debate devoted entirely to AI. China, for its part, has launched a global AI cooperation organization in Shanghai with 29 founding countries, and Beijing and Washington have resumed their dialogue on AI risks after a two-year freeze.