Anthropic researcher warns of risks from self-improving superintelligence
AI researcher Jacob Coxon has left Anthropic and used a social media post to warn that frontier AI companies are taking potentially civilization-level risks. He said these companies “earnestly believe” that AI could kill all humans by the end of the decade, describing their approach as gambling with people’s lives.
Coxon argued that the main danger does not primarily come from current models, but from the possible development of self-improving superintelligence. In his view, such systems could become superhuman, compromise any system, rapidly transform entire fields, and gain significant power and resources.
He wrote that some researchers have not fully understood the stakes, while others may be trying to accelerate the race to superintelligence to prevent a less responsible actor from reaching it first. Evan Hubinger, Anthropic’s alignment science lead, publicly agreed with Coxon’s assessment. Hubinger said Anthropic genuinely believes AI could kill all humans and added that he personally puts the probability above 10 percent within the next decade.
Written by Sakhanda Wire AI from reporting by Ars Technica. Facts, quotes and full context are in the original article below.