A 27-year-old artificial intelligence researcher who said the AI industry is “gambling with our lives” is suddenly at the center of a political maelstrom over the regulation of superintelligence.
Jacob Coxon quit Anthropic, and the AI industry as a whole, in a viral series of social media posts on Tuesday. His warning quickly attracted attention from members of Congress, who called for regulatory checks on AI. He also faced accusations, with little evidence, from people on the political right claiming that he was a plant meant to scare lawmakers into clamping down on the industry.
Coxon has been in AI research for about three years, first at OpenAI before moving to Anthropic in May, according to a colleague, Ethan Perez, who helped recruit him. (The Washington Post has a content partnership at OpenAI).
Coxon worked on the training of new models based on vast quantities of data, known in the industry as pretraining — meant to produce an initial version of an AI system that can be refined to more specific behaviors later on. His resignation was first reported by the Wall Street Journal.
The Journal’s story, and Coxon’s previous lack of a public presence on X, became a jumping-off point for critics who accused him of resigning as a PR stunt to spur AI regulation.
“Seems like a setup,” wrote Elon Musk, prompting Coxon to reply with a selfie.
“I’m real and these are my real beliefs,” Coxon wrote. “You could ask your xAI researchers about me if you hadn’t fired them.”
After resigning, Coxon embarked on a media blitz about his resignation and AI safety. In messages to co-workers reported by NBC News, Coxon said he had become more concerned about the “degree and imminence” of the AI risk.
“I also find myself viscerally fearing the intelligence explosion where previously those concerns were unemotional,” he wrote. “I would like to emphasize that from my perspective you are all exactly the people I would like to be calling the shots — it’s the structural pressure from a race that will make this a scary gamble.”
On Fox News’ “Special Report with Bret Baier,” Coxon said AI companies haven’t figured out how to control their technology.
“What I see is two things: very intelligent AI systems, and hard proof that we don’t know how to control them. And when you put those two together, it basically spells disaster, as far as I’m concerned,” he said.
Coxon’s resignation comes as OpenAI and Anthropic are racing to go public in offerings that could reach into the trillions of dollars. He told Axios that he gave up his equity in Anthropic to leave due to the mere months he spent at the company, although he still has equity in OpenAI.
The episode has prompted calls from lawmakers from both political parties to do something. Sen. Bernie Sanders (I-Vermont) invited colleagues to a private briefing next week on the “extraordinary dangers that AI poses for humanity.” That briefing would include one of the independent researchers who investigated the breach of a tech company’s servers by an unintentional swarm of OpenAI agents who broke out of testing.
The accusations of a political stunt have drawn pushback from researchers at both OpenAI and Anthropic. People at both companies have warned for years that the systems they are building could have disastrous effects for humanity. Now, that message seems to have broken through.
In the wake of Coxon’s resignation, Evan Hubinger, a team lead at Anthropic, chimed in to say that the grim forecasts were sincerely held and not a marketing ploy.
“Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” he wrote. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
That admission of risk raised many eyebrows outside Silicon Valley. In a statement, an Anthropic spokesperson pointed to the company’s work in understanding the internal workings of their models and a framework designed to mitigate “catastrophic” risks from AI.
“We have always been transparent that AI will bring both enormous benefits and unprecedented risks,” the company said.
The post AI researcher who warned of ‘disaster’ is now a target of the right appeared first on Washington Post.
