Robert Wright publishes the NonZero Newsletter and is the author of, most recently, “The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning.”
As Anthropic approached this week’s public stock offering, the company persisted in its famously paradoxical messaging — warning about the dangers of rapidly advancing artificial intelligence while telling us how rapidly its artificial intelligence is advancing.
Earlier this month, in an amendment to its S-1 filing with the Securities and Exchange Commission, Anthropic said it had now reached the threshold described in its safety policy as a “dramatic acceleration” of AI progress attributable to the growing role of AI itself in creating new models. In the original filing, it had warned that this kind of acceleration could “cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures.” Also: Anthropic may not be able to identify “a significant acceleration in our rate of development until after it has occurred.”
So why doesn’t Anthropic just … slow down?
One answer can be found in an essay posted by CEO Dario Amodei two months ago, an essay that emphasized the value of slowing down — and that some people took as a promise that his company would slow down. Maybe these people didn’t read the fine print. More than two dozen paragraphs after declaring that “we must slow the pace at which we improve the capabilities of AI models,” Amodei added that American AI labs shouldn’t slow down enough to endanger “the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party.”
Amodei’s concern with winning the AI race against China is shared by many in Silicon Valley and Washington. That’s one reason for the near-consensus that any significant slowdown, such as a pause in the large training runs that create new generations of AIs, must be internationally coordinated.
But for political and geopolitical reasons, this prospect seems remote — all the more so now that the scandal that put Nvidia CEO Jensen Huang out of a job has also put wind in the sails of U.S. China hawks and raised the political cost of cooperative overtures toward Beijing by President Donald Trump.
The time may come when an effective slowdown indeed requires international coordination — and certainly the AI challenge will eventually demand various international policies. But right now, a national slowdown — such as a government-mandated pause of large training runs — is in the U.S. interest. It would not pose the threat to American AI supremacy, or to American economic vibrance, that many claim, and it could pave the way for the kind of international cooperation on AI that now seems distant.
Though China’s best AI models are said to be six to nine months behind American models — and a straightforward comparison of capabilities suggests as much — there are two reasons that’s misleading.
First, as Amodei has tirelessly pointed out, leading Chinese AI companies rely partly on the “distillation” of the strongest American models. They give these models lots of prompts, record the responses and then use those input-output pairings as training data. Distillation saves time and computing power and, perhaps most important, can capture the practical benefits of research advances made in American labs. And it works.
So, to the extent that Chinese AI progress depends on distilling the best American models, that progress would slow down in response to an American slowdown.
Meanwhile, even if the gap between the most lucrative American models and rival models narrowed somewhat, a hidden asset of frontier American AI companies would help ensure their commercial success: Anthropic and OpenAI are known to have unreleased models that are much more powerful than their released models — and that, since they aren’t available for distillation, are well ahead of Chinese unreleased models. (The Washington Post has a content partnership with OpenAI.)
These models will supercharge a little-noted part of both companies’ business strategies — developing new drugs and other patentable inventions and then profiting from the intellectual property. Anthropic has bought two biotech companies this year, and OpenAI and Anthropic are collaborating with big pharmaceutical firms, as is Google. The stunning math breakthroughs recently made via unreleased models — in great profusion at OpenAI and on a smaller scale at Anthropic — hint at the competitive advantage such models could confer in other domains.
The commercial potential of unreleased models helps explain why some people suspect that OpenAI and Anthropic would actually welcome a pause if it applied to all their American rivals. These suspicions may or may not be right, but their plausibility supports the expectation that a government-mandated pause in large training runs would permit continued commercial dynamism at the American AI frontier.
And, as Wharton professor Ethan Mollick has noted, if the creation of new generations of models stopped now, the productivity gains from AI in the broader economy would accumulate for years as American companies figured out how to adapt the technology to their needs.
The advantage enjoyed by America’s leading AI companies over Chinese rivals will be strengthened by some likely consequences of the Nvidia scandal: stricter enforcement of rules banning the export of high-end chips to China and new rules that limit the ability of Chinese companies to rent processing power from data centers outside of China. In addition to handicapping Chinese AI companies, this tightening could, by shrinking the market for advanced chips, put downward pressure on the prices U.S. companies pay for them.
In the long run, the most important by-product of a national pause could be an international pause. Many Chinese elites, in and out of government, think that calls in Washington and Silicon Valley for an international slowdown are just an attempt to solidify America’s AI lead — a crafty complement to the chip export controls that explicitly serve that goal. A unilateral U.S. pause could help dispel these suspicions and, as the risks of AI become clearer to both governments, pave the way for an international slowdown.
But if there’s no slowdown of any kind in the coming months — if U.S. frontier labs continue to advance at their current pace, while newly tightened chip export restrictions hem China in more fully — the result could be explosive. The bigger the gap between Chinese and American AI, the more threatened Beijing will feel. If America’s AI advantage seems to be approaching a point of clear strategic dominance, preemptive Chinese action could well ensue.
This could mean cyberattacks on U.S. data centers or AI labs, missile strikes on the Taiwanese factories that make the chips China isn’t allowed to import, or various other assaults that could lead to a war between nuclear powers.
The race toward artificial superintelligence may be one of those rare races in which slowing down is the only way to come out a winner.
The post Anthropic says AI should slow down. Washington should make it. appeared first on Washington Post.




