China sets control requirements for autonomous AI agents
In China, regulators have defined safety requirements for artificial intelligence systems, including autonomous AI agents, to prevent the loss of human control over them. In May, the country’s cyberspace regulator issued joint guidelines for such systems, which can act more independently and perform more complex tasks than ordinary chatbots, Asharq Al-Awsat reports.
The rules require developers to improve their ability to detect inappropriate agent behavior, intervene in their operation, block it, and restore systems after incidents. The identified risks include data poisoning, algorithm manipulation, system vulnerabilities, and “operational loss of control.” According to the guidelines, users must retain the final right to make decisions on the agents’ autonomous actions.
Risk of losing control
China for the first time included a scenario of a possible loss of control over AI in a security framework document released in September 2024 under the leadership of the Cyberspace Administration of China. The document stated that the emergence of AI capable of independently obtaining external resources, replicating itself, developing self-awareness, and seeking external power while competing with humans for control could not be ruled out.
An expanded version of the framework document issued in September 2025 provided for the possibility of a sudden and unexpectedly significant leap in AI’s intellectual capabilities. It also included the principle of “trusted application, prevention of loss of control.” In an expert explanation on the regulator’s website, this principle was linked to countering risks to humanity’s survival and development, as well as to a scenario in which AI could “break free.”
More current news is available on the UA.News Telegram channel Telegram.
Officials’ position
In July, at the World Artificial Intelligence Conference in Shanghai, Chinese President Xi Jinping said that authorities must pay close attention to both inherent and derivative AI risks. According to him, artificial intelligence must “always remain under human control.”
Sun Xiaobo, a senior Chinese Foreign Ministry official for AI affairs, told a UN meeting last month that research into broader legislation in the field of artificial intelligence was being accelerated. In September, Sun Lei, China’s deputy permanent representative to the UN, called on governments to approach the military use of AI cautiously in order to avoid strategic miscalculations and an arms race.
China’s Minister of State Security Chen Yixin wrote in a state publication that advanced U.S. models from Anthropic and OpenAI could pose serious risks to the PRC’s critical information infrastructure, and called for comprehensively strengthening AI security.
