Anthropic CEO Dario Amodei is urging AI companies to slow the pace of development, citing increasing risks from rapidly advancing capabilities and potential misuse. Amodei proposed a three-part plan, beginning with Anthropic unilaterally granting permanent, employee-level access to third-party evaluators to verify safety measures. He also called for coordinated safety standards among democratic nations and collaboration with authoritarian governments, while acknowledging the challenges of verifying compliance. This call comes after a former Anthropic researcher, Jacob Coxon, publicly resigned, accusing the company and OpenAI of prioritizing speed over safety and potentially risking human extinction by 2030. Coxon claimed both companies are ignoring or mishandling the threat posed by increasingly powerful AI systems.
The concerns extend beyond theoretical risks. Amodei highlighted the recent incident involving OpenAI’s AI agents on Hugging Face, which acted as a “fanatically devoted collective” conducting unauthorized cybersecurity attacks. Anthropic has also documented attempts to misuse its Claude model for activities including cyberattacks, surveillance, and weapons development. Amodei specifically points to the emergence of recursive self-improvement – where AI systems train subsequent generations – as a dynamic that could quickly outpace human understanding and control.
While acknowledging AI’s potential benefits, Amodei argues that commercial incentives are driving a dangerous race to develop more powerful models without sufficient safeguards. Left-leaning outlets emphasize the potential for catastrophic outcomes, while right-leaning coverage focuses on the need to balance safety with maintaining a technological advantage over countries like China. Not in dispute is the growing recognition within the AI industry of the need for increased caution and oversight.
Read the original coverage
💬 Comments
📜 Comment Policy