A difficult question lurks in the latest warnings from the people building artificial intelligence. When you believe your company may be creating a danger to society, is it your duty to stay and change it, or to leave and sound the alarm?
The question sharpened last week when former Anthropic and OpenAI researcher Jacob Coxon warned that frontier labs were racing toward self-improving superintelligence and “gambling with our lives.”
Coxon is the latest AI insider to walk away while raising concerns about where the technology is headed. Economist Albert O. Hirschman famously described the choice facing people who believe an institution is going astray as “exit” versus “voice”: leave, or stay and try to change it from within. Staying offers access and influence. Leaving brings the freedom to speak publicly. But timing complicates both choices. The longer someone participates in a system they later describe as dangerous, the harder it becomes to separate principled dissent from retrospective absolution.
The people still running the frontier labs face a different version of the problem. Anthropic CEO Dario Amodei published an essay this weekend arguing that AI companies must slow the advancement of their most powerful models so safety work can catch up. But the limits he proposes depend on industry-wide coordination rather than Anthropic acting alone, an acknowledgment that safety concerns run up against commercial and geopolitical competition.
In an exclusive interview with Fortune editor-in-chief Alyson Shontell, OpenAI CEO Sam Altman similarly argued for pacing AI developmentand acknowledged that AI escaping human control is possible. He also suggested that leading AI companies are discussing collective action. For the people running the leading AI companies, the dilemma shifts from whether to stay to whether to slow down. What does it mean to warn that the technology you are building could become catastrophically dangerous while continuing to compete to make it more powerful? At some point, a warning has to lead to a decision about what a company will actually do differently because of the risk.
Altman is now wrestling publicly with where that point lies, who should set the limits, and whether the companies racing to build the most powerful AI can agree to slow down together. Shontell presses him on those issues in their wide-ranging conversation. Watch it here.
Ruth Umoh [email protected]
The post AI’s leaders have a problem: They fear the race they’re trying to win appeared first on Fortune.
