An Anthropic engineer resigns due to fear of AI: "It could kill us all by the end of this decade"

BarcelonaA British engineer at Anthropic, who has also worked for OpenAI –the creator of the popular ChatGPT–, has resigned because he says that both companies "are not acting responsibly" and are "playing with our lives". Jacob Coxon, a software engineer who various media outlets have identified as one of the developers of GPT-4 and GPT-4.5, has stated in a thread on his X account that the people behind the exponential development of AI "believe that it could kill us all by the end of this decade", and that "there is no other human activity that poses this level of threat".

Coxon has not been the only one to speak out in this direction: Anthropic's head of alignment science, Evan Hubinger, responded to Coxon's thread indicating that "he is right." "We wholeheartedly believe that AI could kill all humans," asserted Hubinger, who added that he believes the chances of this ending up happening are higher than 10%. "I think Anthropic is trying its best, but we still don't have a plan to solve alignment for superintelligence and we are clearly not on the right path to do so," he explained. It should be noted that according to Hubinger, the concern is not about "current models," but about the "superintelligence that emerges from self-recursive improvement."

For Coxon, the power of technology is being underestimated, but AI projects "will soon be superhuman systems that will be able to hack anything, revolutionize all fields, and gain a lot of power and resources." In his view, currently both OpenAI and Anthropic are aware of this situation, but he sees a difference: the former "may not have deeply internalized the challenges for civilization," while the latter understand them well but "are trapped in the race to be the first."

Coxon explains that he sees no other way out than coordination between the different AI projects and the companies involved in them, but he believes it is not happening. For this reason, he advocates for a temporary ban on improving the capabilities of the models. "Do you want to promote superintelligent reinforcement learning [RL in technical jargon] without a rigorous understanding of its mind? Will you close your eyes, because it is already happening anyway, or will you take advantage of this moment to defend different conditions?", concludes Coxon.