The chief executive of Anthropic called for a global slowdown of artificial intelligence development in a 3,800-word essay on Saturday, just days after one of the company’s employees quit over concerns about the safety of the technology.
Dario Amodei, who co-founded Anthropic to focus on securely and carefully building A.I., wrote that while he believed the technology could bring many benefits, it was advancing at too quick a pace for researchers to continue safely.
“Over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up,” Mr. Amodei said. “We must slow the pace at which we improve the capabilities of A.I. models. Progress will still seem fast, and we must make wise use of the time we gain.”
His call for an industrywide pause follows months of escalating concerns across the technology industry as A.I. model development has made significant strides at companies like OpenAI, Google’s DeepMind and Anthropic.
Tensions rose further after the leading A.I. labs discovered security breaches carried out by their own programs without their knowledge. In one incident this year, OpenAI’s systems breached an A.I. start-up called HuggingFace, which was not detected until HuggingFace informed OpenAI weeks after it had happened. Since then, A.I. researchers have conferred to discuss what to do.
On Tuesday, Jacob Coxon, an Anthropic researcher, resigned and posted a series of social media posts about how the leading A.I. labs were not responsibly developing the technology and were thus gambling with people’s lives, setting off a furor. Other A.I. researchers have called on their leaders to pause A.I. development, while some are forming splinter groups to raise more awareness about the risks of A.I.
A.I. leaders are girding themselves for a day when an A.I. system can rely on itself to grow more advanced rather than require the aid of human researchers. This dynamic, which computer scientists call “recursive self improvement,” is what some top A.I. developers believe may lead to the software spinning out of their control.
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” Mr. Amodei said.
For years, many of the top A.I. labs have been locked in bitter competition with one another. But as these systems have advanced, there has been some broader agreement among executives at rival companies that something needs be done.
Last month, as OpenAI prepared to release an advanced A.I. model called Astra, the company said it had intentionally slowed down its pace of development, citing advancements in cybersecurity capabilities. In July, Google’s Demis Hassabis published an open framework for what he believed would be a safe way to cooperatively develop next generation A.I.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.)
There are few rules on A.I. development, though President Trump signed an executive order in June asking for A.I. labs to voluntarily submit their models for review before public release.
In his essay on Saturday, Mr. Amodei suggested actions that the industry might take to slow down the pace of development.
Mr. Amodei said all A.I. labs could agree to third-party technology assessments from “embedded evaluators,” or outside specialists who can verify best safety practices across companies. He also suggested that countries with democratic governance systems coordinate to create safety standards, which could take the form of regulatory action.
He added that it would probably require a global effort working with other nations, including authoritarian ones, to properly coordinate a slowdown.
Mr. Amodei stressed in his essay that he still finds A.I. capable of bringing “incredible benefits” to humanity, including potentially curing diseases and accelerating economic growth. But even so, Mr. Amodei said the risks of A.I. were too great to not proceed with extreme caution.
“The measures I propose to advance the frontier at a safe pace will not be easy,” Mr. Amodei wrote. “But I believe we owe it to humanity to try.”
The post Anthropic C.E.O. Calls for A.I. Slowdown appeared first on New York Times.
