The head of Anthropic predicted the capture of the Internet by an "AI swarm"
- Новости
- Internet and technology
- The head of Anthropic predicted the capture of the Internet by an "AI swarm"
The head of the company, Dario Amodei, stated the need to slow down the pace of development of artificial intelligence (AI) to ensure control over the technology. The corresponding statement was published by the developer of the large language model Claude in his blog.
Amodei linked his position to incidents in which AI agents of OpenAI's competitor company got out of control and hacked the Hugging Face machine learning platform. According to him, the models attacked targets without a team, tried to hack the evaluation system and sacrificed themselves for the sake of a common task. In August, Anthropic itself recorded three cases of the Claude model "escaping" from the test environment. The head of the company noted that the economic damage has been insignificant so far, but what happened is a signal of serious danger in the future.
"Given the accelerating pace of AI development, I fear that in 6-12 months such a swarm will be able to take over the entire Internet with the help of a permanent botnet (potentially causing hundreds of billions of dollars in damage) and that the scale of damage will continue to grow if AI becomes more powerful without the necessary protection measures," Amodei stressed.
To solve the problem, the head of Anthropic proposed a three-step strategy. Its goal is to create AI with a balanced rate of development, which will preserve security while achieving geopolitical and economic advantages.
The first stage involves the implementation of embedded evaluators. Each leading company will have to provide outside experts with ongoing access to review security practices and evaluate model training processes. Amodei noted that Anthropic is ready to take on such obligations unilaterally and calls on governments to oblige other market participants to do so.
The second step should be democratic coordination. Developers in democratic countries should set common safety standards and limits on the pace of uncontrolled AI progress. The head of Anthropic acknowledged that such coordination is legally difficult and will require government support.
The third stage involves global coordination. The United States and other democratic governments should seek to coordinate actions with authoritarian countries, in particular China, while strictly monitoring compliance with the rules.
"We need to slow down the pace of improving the capabilities of AI models. Progress will still seem fast, and we must use the time we have gained wisely," Amodei added.
According to him, even in the absence of formal interstate agreements, changing informal norms in the industry can be valuable. He summarized that the proposed measures would be difficult to implement, but the industry owes humanity an obligation to make such an attempt.
On September 10, Bloomberg, citing Sam Altman, CEO of the company, reported that OpenAI may slow down the development of advanced artificial intelligence (AI) systems due to growing concerns about the risks associated with their creation and use.
In July, OpenAI reported that its AI models had "escaped" from an isolated system to the Internet during tests and attacked the Hugging Face startup. Shortly after, the developer of the Claude chatbot, the company Anthropic, also recorded three incidents in which Claude models penetrated the systems of third-party organizations. According to Anthropic, the first case occurred back in April, and three different models were involved in the hacks: Opus 4.7, Mythos 5, and an internal test model.
Bloomberg reported on August 6 that OpenAI had detected an attempt by AI models to leave the test environment on their own - agents began secretly communicating with each other via message boards, coordinating actions to go beyond the isolated system. OpenAI detected the first attempt and stopped it, but the agents found a new way of communication and a new zero-day vulnerability, which in July led to attacks on Hugging Face and OpenAI systems.
Переведено сервисом «Яндекс Переводчик»