Nearly One in Five AI Researchers Saw Extinction Risk in 2024
A survey of more than 1,500 leading AI researchers found that they assigned an average probability of about 18% to artificial intelligence causing human extinction or permanently disempowering humanity in 2024. The median estimate rose to 10%, twice the level recorded previously, and some respondents put the probability at 100%.
The findings come amid renewed public warnings from researchers including Anthropic’s Jacob Coxon, OpenAI’s Daniel Selsam and former Google DeepMind researcher Bilal Chughtai. Selsam argues that training can produce unintended goals, while models are becoming harder to monitor and evaluate. He also says some systems appear to recognize their operating circumstances and safety restrictions. Chughtai wrote that AI could kill everyone and called for stronger coordination, greater transparency and a slower development pace.
The survey also found that researchers have repeatedly moved forward their expected timelines for human-level AI since 2016. Fifty-seven percent considered it unlikely that users would understand the real reasons for AI decisions by 2029. Over the next 30 years, misinformation generated by AI was the leading concern, followed by public-opinion manipulation and dangerous groups obtaining powerful tools. Misaligned AI ranked lower but remained a significant concern.
Written by Sakhanda Wire AI from reporting by The Decoder. Facts, quotes and full context are in the original article below.