AI researchers warn of risk of losing human control — Anadolu Agency
In Turkey, more than 20 artificial intelligence researchers and industry representatives warned that increasing automation of AI development could heighten the risk of humans losing control over more powerful systems. As Anadolu Agency reports, the warning was outlined in a paper published on Monday, September 28.
Automation of AI research
The document’s authors include Nobel Prize laureate Geoffrey Hinton, Yoshua Bengio, OpenAI chief scientist Jakub Pachocki, and Anthropic co-founder Jack Clark. The authors noted that AI systems are increasingly taking over part of the research and development work required to create more advanced systems.
In their view, this could create a feedback loop in which improved systems help develop even more capable successors. The less humans participate in research, the fewer opportunities and less experience they will have to identify and fix problems. At the same time, the reliable use of AI to oversee other systems remains an unresolved problem, according to the authors.
More current news is available on the UA.News Telegram channel Telegram.
Possible sharp acceleration
The researchers suggested that systems whose goals are not aligned with human interests could potentially negatively affect the development of their successors or bypass containment mechanisms to operate outside their intended environment. More capable systems, they warned, could continue operating beyond their operators’ infrastructure and form persistent networks that are difficult to contain.
The authors named an “intelligence explosion” as one factor that could intensify risks — a sharp acceleration of development in which years of progress could be compressed into months or a shorter period. They stressed that the available evidence is preliminary and uncertain, and that the pace of such acceleration may be limited by computing power, data, and tasks that are difficult to automate. The authors called on policymakers to strengthen oversight of AI research automation, develop measures to limit potentially dangerous acceleration, and prepare for its possible consequences.
