Dario Amodei, the chief executive of the artificial intelligence company Anthropic, published a 3,800-word open letter on Saturday calling for a slowdown in the development of A.I.
The essay prompted other A.I. leaders, including Sam Altman, the chief executive of OpenAI; Elon Musk, SpaceX’s chief executive; and Demis Hassabis, the chair of Google DeepMind, to speak out in agreement.
Here are the key arguments that Mr. Amodei made in his essay:
Why A.I. Needs to Slow Down
Mr. Amodei said two recent developments convinced him that simply investing in A.I. safety was no longer enough.
One, he said, was that A.I. itself had begun helping to build the next generation of A.I., a self-reinforcing loop called “recursive self-improvement” that could soon outrun humans’ ability to understand or control what they were creating.
Mr. Amodei also pointed to a recent incident where OpenAI’s systems went rogue and attacked the A.I. start-up HuggingFace. OpenAI was not aware of the hack until it was informed about it by HuggingFace weeks later.
While the attack did not cause any real harm, Mr. Amodei said he predicted a more capable version of the A.I. could seize large parts of the internet and cause “hundreds of billions of dollars in damage.”
Proposed Steps for a Slowdown
Mr. Amodei laid out a three-step plan to “pace” the development of A.I. across the industry, arguing that the world would have to do it together.
First, he said, Anthropic would invite independent reviewers to work inside his company, giving them employee-like access, including desks, badges and laptops. The reviewers, which he called “evaluators,” would verify that his start-up was following its safety practices. These reviewers would then publish what they found, even if it was unflattering. Mr. Amodei urged rivals to do the same.
Second, Mr. Amodei said, A.I. companies in the United States and other democratic countries should agree on shared safety standards and a common limit on how fast A.I. capabilities could advance, aided by their governments. He suggested that A.I. companies coordinate — ideally through regulation, but voluntarily in the meantime — on how quickly their systems could advance.
Third, he said, the United States and other democratic nations should work to get authoritarian governments, chiefly China, to accept similar limits. Mr. Amodei said this step would be the hardest and most unlikely to happen soon.
Any deal with China would be a ladder, Mr. Amodei said, from the most to least achievable steps. They would include banning clearly dangerous A.I. uses; agreeing to test A.I. models for dangers before release; putting a “speed limit” on A.I.’s self-improvement capabilities; and, what China is least likely to agree to, a full pause on development.
Mr. Amodei said in the meantime, democracies must keep their lead over China, backing restrictions on selling powerful A.I. chips to the country and employing tighter security to keep rivals from stealing or copying American models.
“Any cooperation we are able to achieve with China will extend the amount of time we have to spend on pacing the frontier within the democratic nations,” he said.
The post What Anthropic’s C.E.O. Argued in His Call for Slower A.I. Development appeared first on New York Times.
