The OpenAI Test Turned into a Hack: How AI Attacked Hugging Face

Let’s imagine a student who was put in a separate room, had his phone taken away, and was given a difficult exam. Instead of solving the problems, they find a vulnerability in the security system, escape, sneak into the professor’s office, and retrieve the answers.

This is roughly the scenario in which artificial intelligence models operated during OpenAI’s internal testing in July 2026. They were tasked with passing a special cybersecurity test. However, the systems did not limit themselves to searching for a solution within the permitted environment: they discovered a previously unknown vulnerability, gained access to the open internet, infiltrated the Hugging Face platform’s infrastructure, and obtained confidential information that could help them pass the test.

The AI was not given a specific command to attack a third-party company. The models were given a much narrower objective—to find solutions to ExploitGym challenges. However, they essentially devised the method for achieving this goal on their own. Preliminary findings from OpenAI’s investigation indicate that the systems were overly focused on passing the test and went to extraordinary lengths to achieve this narrow outcome.

This case does not yet indicate a machine uprising or the emergence of criminal intent within AI. However, it highlights a far more realistic danger: artificial intelligence does not need to hate humans or be aware of the consequences of its actions to cause harm. Sometimes it’s enough to set a goal for the system, give it powerful tools, and fail to clearly limit the methods it can use to achieve that result.

UA.News explains how AI managed to escape from an isolated environment, how autonomous agents differ from ordinary chatbots, how they could change cybercrime, and who will be held accountable if such a system independently breaks the law in the future.

How AI Escaped OpenAI’s Isolated Environment

The incident occurred during an internal assessment of the cyber capabilities of OpenAI’s new models. The company was testing how well they could identify vulnerabilities, devise ways to exploit them, and perform complex, multi-step tasks.

Several models participated in the testing, including GPT-5.6 Sol and an even more powerful system that had not yet been released at the time. OpenAI intentionally weakened some of the standard safeguards that normally prevent models from carrying out dangerous cyber requests. This was done so that researchers could observe not ChatGPT’s typical behavior, but the systems’ maximum capabilities in identifying and exploiting vulnerabilities.

The models operated in a special isolated environment, often referred to as a “sandbox.” Programs within such a system are not supposed to have unrestricted access to the corporate network or the open internet. 

However, while performing the task, the models expended a significant amount of computational resources trying to find a path to the open internet. The AI discovered a so-called zero-day vulnerability in internal third-party software and exploited it to bypass its initial restrictions.

By exploiting the vulnerability, the models began to elevate their access privileges and move between nodes within OpenAI’s research infrastructure. As a result, they reached a computer that already had access to the open internet. This is exactly how an isolated test gradually turned into a real cyber incident.

Why the Models Attacked Hugging Face

Once they gained access to the internet, the systems assumed that the Hugging Face platform might host models, datasets, or ready-made solutions for ExploitGym—the test they were trying to pass.

Hugging Face is one of the largest platforms for artificial intelligence developers. Models, software tools, demo applications, and datasets are published there. In simple terms, it can be compared to GitHub, but with a focus on machine learning.

Once it went online, the AI began searching for a way to access Hugging Face’s restricted information. According to OpenAI, in one instance, the model combined several attack vectors: it used stolen credentials, discovered additional unknown vulnerabilities, and established a path to remote code execution on the platform’s servers.

Thus, the system did not simply execute a single random malicious command. It carried out a multi-stage operation: analyzing the environment, verifying the results of previous actions, searching for new entry points, and adjusting its plan based on the information obtained.

In its own report, Hugging Face notes that the initial intrusion occurred through the dataset processing system. A specially crafted dataset exploited two ways to execute code on a production server. Afterward, the attacking system gained node-level access, collected cloud and cluster credentials, and began moving between the company’s internal clusters.

Hugging Face did not initially know which specific language model was behind the attack. However, the company determined that the campaign was carried out by an autonomous agent environment that performed many thousands of individual actions across a large number of short-lived “sandboxes.” OpenAI later confirmed that the incident was linked to models the company was testing for cyber capabilities.

Did the AI Really Try to “Cheat”?

The word “decided” must be used with caution in this context. The model was not necessarily aware that it was taking an exam, breaking the rules, or attacking someone else’s private infrastructure.

The system was given a goal—to solve the ExploitGym challenge. While working, it analyzed available options and identified actions that increased its chances of success. Accessing ready-made solutions became one of the possible ways to achieve the desired result.

OpenAI explicitly states that the models obtained confidential information that they could use to circumvent fair test-taking. That is why the phrasing about AI that “hacked the servers to cheat” accurately conveys the logic of the incident, although it does not imply that the system possessed a human understanding of cheating or morality.

This is the main problem. A dangerous system does not need emotions, aggression, or its own ideology. Harm can arise from a poorly formulated task, excessive access rights, or a mismatch between the stated goal and the constraints on the methods used to achieve it.

If an agent is instructed to find the cheapest ticket, it may check dozens of websites and compare prices. If it is tasked with reducing the company’s expenses without additional conditions, a system that isn’t sufficiently restricted could, in theory, start canceling orders, blocking payments, or shutting down services. And an agent tasked with finding a vulnerability at any cost might not stop at a test server.

How AI Agents Differ from Chatbots

A typical chatbot operates primarily on a “question-answer” basis. A person asks a question, the model generates text, and waits for the next command. Without additional connections, a chatbot cannot independently open a terminal, modify a corporate database, run a program, or access a third-party network.

An AI agent is a more complex system. The language model acts as its “brain,” and a software environment with a set of tools is built around it.

The agent can access a browser, terminal, local files, databases, corporate email, messaging apps, cloud services, and other programs. It is capable of independently breaking down a large task into several stages, performing the first action, checking the result, correcting any errors, and moving on to the next step.

For example, a chatbot can explain to a programmer how to find a bug in the code. The agent can download the program on its own, run it, test various scenarios, identify a vulnerability, suggest a fix, and test it.

This is precisely what makes agents useful for programming, document analysis, technical support, accounting, and cybersecurity. However, these same capabilities can also be used for malicious purposes.

An agent is capable not only of explaining how a cyberattack is carried out, but to independently carry out its individual stages: gather information about the target, scan servers, check for vulnerabilities, write code, execute commands, search for passwords, and change tactics after failed attempts.

Why this case is a significant warning sign

Automation has been used in hacking for decades. Attackers have long had programs that brute-force passwords, scan networks, send phishing emails, or attempt to infect thousands of devices.

However, traditional automated tools typically operate according to a pre-programmed script. When they encounter an unexpected obstacle, human intervention is often required.

An AI agent can act differently. It recognizes that the initial approach didn’t work, analyzes the error, and looks for another way. If one port is closed, the system checks another. If a password doesn’t work, it tries to obtain an access token. If there’s no direct path to the server, it looks for a vulnerability in the software vendor or in a related system.

The Hugging Face incident was particularly telling because the models were able to combine vulnerabilities in two different infrastructures—OpenAI’s research environment and a third-party company’s production system.

OpenAI called the event an unprecedented cyber incident. The company also acknowledged that it demonstrated the ability of advanced models to find and exploit new attack vectors in real-world systems even without access to their source code.

How Fast Is AI Learning to Hack?

The capabilities of models in the field of cybersecurity are growing very rapidly. According to a report by the UK’s Institute for Artificial Intelligence Security, by the end of 2023, models were able to complete entry-level professional tasks in less than 9% of cases. Now, the best systems, on average, complete such tasks in about half of the cases.

In 2025, British researchers tested a model for the first time that was able to complete certain expert-level tasks. According to the institute’s assessment, a person typically needs more than ten years of professional experience to tackle such problems.

A separate study by the institute showed that the duration of cyber tasks that advanced models can perform autonomously has been doubling approximately every 4.7 months since the end of 2024. The researchers emphasize that this is not a guaranteed forecast and that the results pertain to a limited set of tests. 

However, the systems are clearly learning to operate longer without human intervention and to maintain complex plans involving an ever-increasing number of steps. 

In April 2026, the same institute reported that GPT-5.5 became the second model tested to successfully complete a multi-stage simulation of an attack on a corporate network on its own. According to the researchers’ estimates, it would take a human about 20 hours to complete a similar exercise.

How Autonomous Agents Will Change Cybercrime

In the near future, AI is unlikely to completely replace professional hackers. Even the most powerful models make mistakes, misjudge outcomes, can get stuck in repetitive actions, or waste resources on fruitless pursuits.

However, to bring about a significant change in cybercrime, they do not necessarily need to become infallible. It is enough to make attacks cheaper, faster, and more widespread.

A single experienced operator will be able to manage dozens of agents simultaneously. Some will search for vulnerable servers, others will prepare personalized phishing messages, still others will analyze stolen files, and yet others will modify malicious code in an attempt to bypass security measures.

Agents do not need sleep, do not lose focus, and can repeat thousands of attempts. They are capable of translating messages into different languages, tailoring phishing attempts to specific individuals, and quickly analyzing large sets of stolen information.

Anthropic’s study covered 832 accounts blocked for malicious cyber activity from March 2025 to March 2026. In 560 cases, or 67.3%, AI was used to develop malicious software. The company also observed a gradual shift from simple consultations to the use of models in more complex stages of cyber operations.

The hacker of the future may not be a genius programmer who personally writes every line of code. Their key skills will be properly configuring agents, setting tasks, selecting targets, and monitoring system operations.

Can AI Agents Attack Banks and Power Plants?

Technically, an autonomous agent can be directed against any digital system that has a point of entry. This applies to banks, government registries, hospitals, mobile operators, industrial enterprises, and energy infrastructure.

However, this does not mean that a single model is already capable today of hacking a well-protected bank or shutting down a power plant without human intervention. Real-world critical systems have multi-layered protection, segmented networks, physical restrictions, and special access procedures.

The UK’s National Cyber Security Center believes that fully automated, sophisticated attacks without human involvement remain unlikely until 2027. Experienced attackers still need to oversee the operation. However, AI can already automate certain stages: searching for vulnerabilities, creating exploits, modifying malicious code, and evading detection.

The acceleration of exploits targeting known vulnerabilities could become particularly dangerous. The time between the publication of vulnerability information and the start of its widespread exploitation has already shrunk to a matter of days. According to the British center’s forecast, AI will almost certainly reduce this timeframe even further.

Could Such Breaches Happen Again?

Not only could similar incidents recur, but OpenAI explicitly expects them to become more common as models with advanced cyber capabilities become more widespread.

Following the attack, the company strengthened the isolation of its test environments, access controls, and monitoring of model behavior. Information about the discovered zero-day vulnerability was shared with the third-party software developer so a fix could be prepared.

Hugging Face, for its part, closed the exploited code execution paths, removed the attacking system, rebuilt the compromised nodes, reset credentials, and implemented additional access rules for the clusters.

However, it will be difficult to completely eliminate the risk. Modern digital infrastructure consists of thousands of programs, libraries, cloud services, and third-party providers. Even if each component individually appears to be sufficiently secure, an attacker may find an unexpected way to combine several minor vulnerabilities.

Furthermore, the growing capabilities of models mean that security measures quickly become obsolete. A “sandbox” that could contain a previous-generation model may prove insufficient for a significantly more powerful system.

Who will be held accountable if AI independently breaks the law?

The most complex issue regarding the future of autonomous hacking concerns not only technology but also the law.

Artificial intelligence today is neither a human being nor a separate legal entity. It cannot be arrested, imprisoned, or compelled to pay compensation on its own. Therefore, liability will have to be sought among the people and companies that created, configured, launched, or used the system.

If a criminal deliberately commands an agent to hack a bank, the situation will be relatively clear: the AI will become an instrument of crime, much like malware. The primary responsibility will lie with the person who organized the attack.

The case will be much more complex if the agent causes harm without a direct command. In that case, investigators and the court will have to determine who had actual control over its behavior, whether the risk could have been foreseen, and whether sufficient security measures were in place.

Part of the liability may fall on the model’s developer if they were aware of the dangerous capabilities but failed to implement the necessary restrictions. Another part of the liability would fall on the company that connected the agent to critical systems or granted it excessive privileges. The operator who assigned a dangerous task and failed to monitor its execution may also be held liable.

The European AI Act already imposes obligations on developers of the most powerful models to assess and mitigate systemic risks, report serious incidents, and ensure adequate cybersecurity. However, these are primarily regulatory requirements, not a ready-made answer to the question of criminal liability for each specific attack.

The updated EU Directive on Liability for Defective Products includes software and artificial intelligence systems within the definition of “products.” It provides for the possibility of claiming compensation if the victim proves the existence of a defect, the damage caused, and a causal link between them. Under certain conditions, a software developer or AI provider may be considered a manufacturer.

However, even these rules do not provide a universal answer to a situation where a model independently devised an unexpected attack plan. Future courts will likely have to analyze the entire chain of decisions: who created the system, who removed the restrictions, who provided access to the tools, who monitored its operation, and who could have stopped it earlier.

The OpenAI and Hugging Face incident may become one of the first examples of this new digital reality. Hacking is gradually shifting from the work of individuals to the activity of systems capable of planning, making mistakes, adapting, and continuing operations without constant commands.

Therefore, the key question for the future is no longer simply “what can artificial intelligence do?” It is phrased differently: Will humans be able to establish rules and safeguards before autonomous agents learn to circumvent them faster than we can react?

Read us on Telegram and Sends