Researchers warn automated AI research could trigger an intelligence explosion
More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio and OpenAI research lead Jakub Pachocki, warn that self-improving AI could cause an “intelligence explosion.” In a research paper, they say there is still substantial uncertainty but argue that automated AI research and development might soon set such a process in motion.
The authors say AI systems already produce most of the code at companies developing them. They could automate the full AI research and development pipeline within years, potentially compressing progress that normally takes years into a matter of months.
The researchers warn that society may be unable to adapt quickly enough, that control over superhuman AI could be lost and that existing power balances among nations, companies and governments could weaken. They call on policymakers to obtain much better visibility into the automation of AI research. The paper follows other warnings, including a recent statement from 42 leading mathematicians about existential AI risks and concerns voiced by AI laboratory employees. Pachocki has also said that no laboratory has solved alignment sufficiently to continue scaling at maximum speed responsibly for much longer.
Written by Sakhanda Wire AI from reporting by The Decoder. Facts, quotes and full context are in the original article below.