AI tech leaders have a lot to gain from striking fear into the hearts of their investors. By painting the tech as an ultra-powerful force that could easily bring humanity to its knees, the industry is hoping to sell itself as a panacea: a remedy to a situation it had a firm hand in bringing about.
Case in point, Anthropic cofounder and CEO Dario Amodei is back with a 19,000-word essay posted to his blog, arguing that “humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”
In light of that existential danger, Amodei attempted to lay out a framework to “defeat” the risks presented by AI — which, by his own admission, may well be “futile.”
“Humanity needs to wake up, and this essay is an attempt — a possibly futile one, but it’s worth trying — to jolt people awake,” he wrote.
Amodei argued that “we are considerably closer to real danger in 2026 than we were in 2023,” citing the risks of major job losses and a “concentration of economic power” and wealth.
However, the incentives to invest in meaningful guardrails simply aren’t there. In his essay, Amodei took a thinly-veiled dig at Elon Musk’s Grok chatbot, which has been swept up in a major controversy over creating nonconsensual sexual images.
“Some AI companies have shown a disturbing negligence towards the sexualization of children in today’s models, which makes me doubt that they’ll show either the inclination or the ability to address autonomy risks in future models,” he wrote.
The CEO also cited the risk of AIs developing dangerous bioweapons or “superior” military weapons. An AI could “go rogue and overpower humanity” or allow countries to “use their advantage in AI to gain power over other countries,” leading to the “alarming possibility of a global totalitarian dictatorship.”
Amodei lamented that we simply aren’t willing to address these risks head-on, at least right now. In its current race to the bottom, the AI industry finds itself in a “trap,” Amodei argued.
“AI is so powerful, such a glittering prize, that it is very difficult for human civilization to impose any restraints on it at all,” he wrote.
“Taking time to carefully build AI systems so they do not autonomously threaten humanity is in genuine tension with the need for democratic nations to stay ahead of authoritarian nations and not be subjugated by them,” he wrote. “But in turn, the same AI-enabled tools that are necessary to fight autocracies can, if taken too far, be turned inward to create tyranny in our own countries.”
“AI-driven terrorism could kill millions through the misuse of biology, but an overreaction to this risk could lead us down the road to an autocratic surveillance state,” he argued.
As part of a solution, Amodei renewed his calls to deny other countries the resources to build powerful AI. He went as far as to liken the US selling Nvidia AI chips to China to “selling nuclear weapons to North Korea and then bragging that the missile casings are made by Boeing and so the US is ‘winning.’”
Plenty of questions remain surrounding the real risks of advanced AI, a subject that remains heavily debated between realists, skeptics, and proponents of the tech. Critics have pointed out that the existential risks often cited by leaders like Amodei may be overblown, particularly as improvements in the tech appear to be slowing.
We should also consider the greater context of Amodei’s verbose warning. The CEO’s company is looking to close a massive, multibillion-dollar round of funding at a valuation of $350 billion. In other words, Amodei has an enormous financial interest in positioning himself as the solution to the risks he cites in his essay.
More on Anthropic: Anthropic Let an AI Agent Run a Small Shop and the Result Was Unintentionally Hilarious
The post Anthropic CEO Warns That the AI Tech He’s Creating Could Ravage Human Civilization appeared first on Futurism.
