For years, the artificial intelligence industry raced to improve its technology as quickly as possible. Now some researchers are increasingly sounding the alarm about the need to slow down.
Jacob Coxon, an Anthropic researcher who had previously worked at OpenAI, said on Tuesday that he had resigned from Anthropic because “neither company is acting responsibly.” The two leading A.I. labs were sprinting to build “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources” without building in appropriate safeguards, he said in a social media post.
Mr. Coxon’s warning echoed concerns that other A.I. experts have voiced in recent months. Some A.I. leaders have long believed that the technology could create doomsday scenarios if left unchecked, and their concerns heightened after an incident in July when OpenAI’s models broke out of its systems and hacked Hugging Face, an A.I. model library.
That same month, more than 1,300 employees at top A.I. companies including Anthropic, OpenAI, Meta and Google’s DeepMind signed an open letter calling on the U.S. government to create rules to slow the pace of A.I. development. In August, more than 100 major tech companies committed to providing their best A.I. models to organizations like hospitals and infrastructure providers so they could prepare for A.I.-powered cyberattacks.
OpenAI also said last month that it would expand safety testing and slow the release of a new model, Astra, which has powerful cybersecurity abilities.
Some critics have argued that A.I. companies are overhyping the technology to make their products appear more valuable. Several lawmakers said on Wednesday, however, that they were taking Mr. Coxon’s warning seriously.
“There are massive implications of a race towards super intelligence,” Representative Anna Paulina Luna of Florida, a Republican, wrote in a social media post in which she called on Congress to convene a special session focused on A.I. regulation. “Not enough of Congress is focusing on this.”
“Safety researchers are resigning, powerful A.I. models are breaking out of their labs, and companies are racing ahead anyway,” Representative Lori Trahan of Massachusetts, a Democrat, wrote in a social media post. “It’s past time for Congress to get off the sidelines and do its job.”
While leading A.I. labs have agreed that the government should place restrictions on the pace at which the technology is developed and released to the public, many companies are fiercely competing with one another and fear falling behind their rivals if they pause.
“They are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk,” Mr. Coxon said in his social media post.
At least one Anthropic employee agreed with Mr. Coxon’s assessment.
“We really do earnestly believe A.I. could kill all humans!” Evan Hubinger, who works at Anthropic on creating A.I. that aligns with human ethics and values, wrote on social media. Mr. Hubinger said he believed the risk of A.I.’s eliminating humanity was less than 10 percent within the next decade.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he added.
Anthropic and OpenAI did not immediately respond to requests for comment about Mr. Coxon’s comments. Mr. Coxon and Mr. Hubinger also did not immediately respond to requests for comment.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.)
The post Anthropic Researchers Raise Alarm Over A.I. Acceleration appeared first on New York Times.
