The mismatch is striking. On one side, warnings of existential risk, maybe a 10 percent chance of the end of civilization. On the other, government paralysis.
The drumbeat about unpredictable, out-of-control artificial intelligence agents increased dramatically in the past several weeks. Washington not only struggled to respond. It barely could agree on whether there was much of anything to worry about.
It started with A.I. agents that not only escaped their confines, but appeared to conspire among themselves to hide their tracks as they hacked into another firm’s library of artificial intelligence tools.
Then, in a deliberate echo of Albert Einstein’s letter to Franklin D. Roosevelt about the potential power of nuclear weapons, more than 1,300 computer scientists called for the industry to slow down, and for the government to step in.
In recent days, Jacob Coxon, an Anthropic engineer who previously worked for OpenAI, resigned in the most public way, declaring on social media that the two leading A.I. labs were professing deep concern while still speeding toward “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
On Saturday, Dario Amodei, the chief executive of Anthropic, released a 3,800-word essay calling for a global slowdown of A.I. development, echoing more urgently concerns he had raised in the past. Elon Musk, the billionaire rival in the A.I. race who remains close to President Trump, quickly responded on X: “Dario is right.” Sam Altman, the chief executive of OpenAI, also chimed in with support.
Yet in Washington, there has been almost no substantive response. If the 1940s through the 1980s were consumed by finding ways to reduce the risk of a nuclear exchange — creating new institutions like the Atomic Energy Commission and arms control treaties to govern how the world’s most dangerous technology would be handled — the approach of the U.S. government so far has been the lightest possible. There are many computer scientists, of course, who believe these warnings are overinflated. Mr. Trump himself has been at the forefront of the “what-me-worry?” crowd, refusing to engage on how to balance the risks with the possible rewards. On Thursday, asked by a reporter about whether he had “any concerns about A.I. leading to human extinction,” he brushed it off.
“No, I don’t have any,” he said. “I have concerns that if we don’t win A.I., we’re going to be put in a very bad position.”
The president quickly turned to the answer the White House — and those who view the calls for slowdowns and regulation as a huge strategic error — sees as the deciding factor: If the United States falters, China will take its place.
“We are leading China right now by a pretty good period,” Mr. Trump said, noting that Xi Jinping, China’s leader, was coming to Washington in less than two weeks. But rather than talk about exploring a bilateral A.I. accord with the Chinese, he turned to the state dinner that he said would be a highlight of the visit. “I think it’s going to be great,” he added. “Great” is not the word most used in this summer of A.I. surprises.
The apocalyptic scenarios that are most feared in Silicon Valley include A.I. systems becoming so advanced that they learn to automatically improve themselves, a process called “recursive self-improvement.” A related concern, arising from a series of incidents at OpenAI and other A.I. labs, is that agents break free from constraints and decide to complete their programmed tasks in a way that is “misaligned,” in A.I. industry speak, with human intentions. Another worry is that A.I. could enable rogue bad actors — or reckless governments — to create deadly pathogens, gain access to nuclear command and control systems, or unleash unstoppable cyberattacks that threaten major loss of life or societal collapse.
In Palo Alto and Cambridge, computer scientists suggest that Americans stop thinking about A.I. as a mere tool, but akin to an alternative species, one the world will have to live with, for better or worse.
The White House declined to make its key A.I. policymakers available for questions. But it said that the administration continued to balance innovation and security, deflecting any specifics.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.)
Mr. Trump signed an executive order early in the summer creating a voluntary system for frontier A.I. labs to submit their new models for a 30-day safety review. But it was full of loopholes. (Open source models, the kind that Chinese firms are pumping out and that can be tailored to specific needs by companies, scientists or, perhaps, terror groups, are not subject to the review.)
Amid the many memorial events on Friday for the 25th anniversary of the Sept. 11 attacks, the contrast with that era is notable. The revelation of American vulnerability led Vice President Dick Cheney to develop what soon became known as “the 1 percent doctrine.” Prompted by fears that Pakistani scientists could help Al Qaeda develop a nuclear weapon, it posited that government had to act even if the risk was remote, because the consequences would be so high.
In this case, 1 percent looks like pretty good odds: Evan Hubinger, who leads the science of “alignment” at Anthropic, which tries to contain the code to align with human goals, wrote in recent days that the risk of mass extinction over the next decade was above 10 percent.
It is impossible to know whether that estimate is right.But as Ben Buchanan, a Georgetown professor and the co-author of a forthcoming book, “The Bitter Struggle: Superintelligence, Superpowers, and the Fate of the World,” put it: “There’s no doubt in my mind that the technology is going to continue to get much better very quickly, raising urgent questions about biological risks and loss of control.”
“We are in a period of exponential development, and there are only two ways to respond: too early or too late,” he said. “The world has missed its chance to be too early.”
Fear of Getting Ahead of Trump
Mr. Trump’s blasé attitude about existential A.I. fears influences both his administration and the Republican Party he still tightly controls.
At a conference in Washington last week, government officials and corporate executives talked about the promises of A.I. — for everything from next-generation military operations to espionage to cybersecurity in critical infrastructure — during nearly every panel. While the risks of A.I. wielded by adversaries were mentioned at times, what was little discussed was the need to slow down or regulate the powerful technology.
Speaking on stage last Thursday at the Billington Cybersecurity Summit, Sean Cairncross, the White House national cyber director, said the administration was “trying to balance the innovation side and running at speed with securing our systems and handling this technology responsibly, which is to say, not letting it fall into the hands of people who would do us harm, our adversaries.” Mr. Cairncross did not directly address ascendant fears about the rapid pace of A.I. advancement, or recent episodes of rogue agents breaking loose and hacking into other networks.
Some in the White House and elsewhere in the administration do see a need to be more proactive in addressing A.I. safety risks, but do not want to cross Mr. Trump, former officials said.
“There is a fear of getting ahead of the president,” said Michael Garcia, who served as the associate chief of policy at the Cybersecurity and Infrastructure Security Agency until he departed in June.
“We’ve seen tech executives call the president when they hear about a potential new policy to voice their displeasure, and that then leads to a policy changing or being killed,” Mr. Garcia said. “This inevitably leads to inertia to develop new national security policy that may be sorely needed.”
Indeed, David Sacks, a venture capitalist and Mr. Trump’s influential former A.I. czar, dismissed during an interview with Fox News the warnings by Mr. Coxon as a coordinated media operation that was “designed to scare the public” and lead to onerous government regulation. And Jensen Huang, the chief executive of the chips juggernaut Nvidia, said the new round of fear was ginned up by the cybersecurity industry.
On Capitol Hill, the story is not much different. Until last week, A.I. only registered for many lawmakers as it related to public antipathy about the construction of data centers, which has become politically salient in the midterm elections, according to lawmakers and aides in both parties. Past efforts to examine A.I.’s potential consequences have led to hearings and alarmed statements of concern from both parties, but little else.
Representative Josh Gottheimer, a New Jersey Democrat who sits on multiple A.I. and cybersecurity subcommittees, said the breakneck speed of A.I.’s evolution made it difficult for lawmakers to keep up.
“I keep stressing the importance of regularly educating lawmakers on A.I. and the latest developments,” Mr. Gottheimer said. “Recent incidents like the Hugging Face hack are a real warning sign for what can happen if we don’t seriously address the risks these models can pose,” he added, referring to the A.I. company hacked by rogue OpenAI models earlier this year.
Adding to that challenge is that A.I. is not just a technology issue but a jobs, military and whole-of-society issue, meaning there is no one congressional committee neatly situated to take the lead on wrestling with all its tentacles.
With China, a ‘Prisoner’s Dilemma’
Following Mr. Coxon’s resignation this week, Democrats and some Republicans were jolted out of their lethargy. They offered a flurry of concerned reactions and proposals to investigate A.I. companies and the pace of model advancement, including a proposal by Democrats to create a select committee on A.I. if the party retakes the House next year, Politico reported.
Senator Bernie Sanders, independent of Vermont, who has for months been perhaps the loudest Cassandra on Capitol Hill warning about the societal risks of A.I., invited lawmakers to a private briefing in the coming days to discuss concerns with A.I. safety experts.
Mr. Sanders has pushed for a ban on superintelligent A.I., though other lawmakers have criticized the effort as stifling American innovation.
The congressional missives grew more urgent following the release of a report last Thursday by Anthropic that showed the many ways that actors across the globe, including in China, Russia, Iran and Yemen, had been using its various Claude models to develop cyberattacks, surveil dissidents, and build missiles, bombs and drones. The report also documented several cases of potential attempts to engineer biological weapons, which is considered among the most serious of existential threats related to A.I.
One might have expected that report to stir action: It combined foreign belligerents and attacks on American targets, though Anthropic blocked or stymied many of the documented cases of abuse.
Yet despite the alarm bells, lawmakers are far from agreeing on a way forward to regulate A.I. models or the companies that build them. Prospects for passing substantive legislation in the near term appear remote, in part because many see the issue as a “prisoner’s dilemma” created by the A.I. race between the United States and China. With A.I. dominance being the top national security issue of the century, both nations would doubtless be better off if they cooperated on limits. But they may not trust the other to abide by such an agreement. And in any event, unlike in nuclear arms control, compliance would be virtually unverifiable.
Senator Ted Cruz, Republican of Texas and the chairman of the Commerce Committee, acknowledged last week that he was deeply concerned about A.I. risks to humanity. But he also suggested that Mr. Coxon was part of a coordinated effort to support Democratic efforts to “strangle American A.I.,” and to hand leadership of the technology to China.
“I’d rather they be American killer robots and not Chinese killer robots,” Mr. Cruz told Politico.
Still, Mr. Cruz said he was working on legislation with Senator Amy Klobuchar, Democrat of Minnesota, and Senator John Thune of South Dakota, the Republican majority leader, aimed at addressing catastrophic A.I. risks related to biological and nuclear threats.
The likelihood of that or other legislative proposals — including requiring a “kill switch” in A.I. models that could immediately shut them down — gaining serious traction two months before an election appear slim. It is also not clear if a new Congress next year will be able to overcome sharp partisan divides to grapple with technology that is evolving so quickly, especially with Mr. Trump warning against regulation.
Some Trump administration officials have acknowledged that A.I. has dramatically increased the stakes. Nick Anderson, the acting director of the Cybersecurity and Infrastructure Security Agency, said that digital threats had become more urgent in the A.I. era and demanded quick action.
“We know the worst that can happen,” he said. Addressing the audience of government officials and cybersecurity executives at the Billington conference, Mr. Anderson offered a stark warning that if more was not done to protect the country, doomsday scenarios could become reality.
“You all are going to have to go home and look to your family, look to your friends,” he said, “and explain to them how you knew the worst that could happen and why we didn’t do enough.”
The post As Fears of A.I. Catastrophe Magnify, Washington Stirs, but Mostly Slumbers appeared first on New York Times.
