This, friends, will be my final column at The New York Times. I am off to new adventures, including publishing a book about the race to create artificial general intelligence and starting a new podcast with my “Hard Fork” co-host, Casey Newton. It has been an honor to spend the last nine years as your tech columnist, testing out all manner of gadgets, selling NFTs and being seduced by unhinged chatbots in the name of journalism.
Before I go, my editor has invited me to share some parting thoughts on the A.I. moment we’re in. So I thought I’d give a brief, honest reflection on how I’m feeling about A.I. these days, and how my reporting from inside the San Francisco tech bubble has left me feeling both hopeful and scared for our collective future.
First, the hopeful part.
I have been surprised and heartened, in recent days, by the way conversations about A.I. risk have taken center stage. Last week, the viral resignation of a young Anthropic employee, Jacob Coxon, did what years of worried opinion essays and A.I. summits had not: focused the world’s attention on the potentially extreme dangers posed by this technology.
The substance of Mr. Coxon’s message was nothing A.I. insiders hadn’t been saying for years. But his timing — just days after new revelations about the autonomous hack of Hugging Face by a swarm of OpenAI agents, at a time of rising concern about A.I. risk more generally — helped carry the message far outside the A.I. bubble.
(I should add my parting, mandatory A.I. disclosure: The Times is suing OpenAI, Microsoft and Perplexity over alleged copyright violations. The companies deny the claims.)
Even more mood-lifting was the response to the public call last weekend by Dario Amodei, Anthropic’s chief executive, for a global slowdown in A.I. development — a step that many A.I. researchers had privately told me they thought was necessary to prevent a catastrophe, but that few had been willing to push for in public.
Sam Altman, Elon Musk, Demis Hassabis and other prominent A.I. leaders quickly echoed Dr. Amodei’s concerns and some of his recommendations, such as his call for putting “embedded evaluators” inside leading A.I. companies to monitor their systems for dangerous behavior.
Normally, these men are the fiercest of rivals. (Dr. Amodei and Mr. Altman, for example, can barely stand each other, and Mr. Musk had an expensive, failed lawsuit against Mr. Altman and OpenAI earlier this year.) But recent, rapid advances in A.I. capabilities, and their shared worry about rogue systems taking over critical infrastructure, seem to have given them common cause. That kind of cross-industry cooperation is going to be critical in the coming months and years, and it was good to see it materialize so quickly.
And to ward off the inevitable conspiracy theories: No, this is not a coordinated psyop, or an industrywide scheme to achieve regulatory capture. Many of these figures have been warning about the existential threat of A.I. for years, long before they had products to sell and companies to run. The news is just that they’re actually being listened to now, after years of being written off as doomsaying Chicken Littles.
Like Dr. Amodei and Mr. Altman, I’ve been nervous about the pace of A.I. progress for years. And until recently, it seemed as if actually doing anything to slow down the race was vanishingly unlikely. But the outpouring of calls from A.I. leaders and rank-and-file employees alike for a global, coordinated pause on A.I. development in the past week has made me reconsider my pessimism.
Maybe we are on the narrow path to A.I. utopia — one in which policymakers and the public are sufficiently alarmed about the risks of building these systems to put some common-sense guardrails in place, and A.I. leaders are temporarily convinced to put humanity’s best interests above their own bottom lines. In that world, a coordinated pause could buy us time to understand and manage the worst of the risks, while continuing to push forward narrower A.I. applications in fields like medicine and science, where everyone basically agrees we should move as fast as possible.
Of course, talking about A.I. risks is only a small part of the battle. A much bigger part is doing something about them — and this is where I’m more pessimistic, and I feel my internal p(doom) needle swinging to a higher number.
Already, we are seeing the conversation among A.I. leaders polarize along predictable lines. After Dr. Amodei’s call to “pace the frontier,” other executives, including Jensen Huang of Nvidia, raced to reassure a spooked public that there was nothing to worry about — that all of the doom-and-gloom talk emanating from the frontier A.I. companies was unscientific hogwash, and that America’s A.I. industry should accelerate rather than slow down. President Trump appeared to side with the accelerationists, dismissing fears of an A.I. takeover as a “hoax” and posting on social media that “the only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT.”
Given his advisers’ disdain for A.I. doomsayers, I am skeptical that Mr. Trump will negotiate any kind of A.I. safety pact with China, or do much to keep A.I. from progressing at breakneck speed. I am also skeptical that Congress — which has a dismal track record of regulating new technologies — will rise to the occasion.
Even today, many lawmakers see A.I. through the narrow prism of other partisan-polarized issues, such as wealth inequality or corporate greed. Expanding their concerns to include the risk of extinction from A.I. isn’t impossible, but it requires moving from the familiar debates over who benefits from A.I. to the more uncomfortable question of whether humans will remain in control of it.
Still, like a certain former president, I am hopeful because these conversations are happening at all. For years, A.I. has been developed by a tiny group of like-minded people — mostly researchers in San Francisco and London — and outsiders have largely been shut out of the most important conversations about its trajectory and impacts.
I have interviewed hundreds of A.I. researchers and executives over the past decade, and I have largely found them to be thoughtful and decent. They are not plowing ahead without regard for the consequences of their actions, as social media executives did in the 2010s, or as tobacco executives did in the 20th century.
Many of them are genuinely fearful that the technology they’re building could imperil or disempower humanity, and have chosen to build it anyway because they feel it is inevitable, and because the alternatives — letting China, or North Korea, or the scoundrels at the A.I. lab down the street build it first — all seem worse.
Still, the A.I. insiders are a cloistered, imperfect group, and they should not get to decide the fate of humanity any more than fast-food executives should get to choose global nutrition standards, or airline executives should get to design the industry’s safety protocols. If this A.I. thing is going to work out — and I’m still hopeful that it might — it will need to be a collective effort. We need more philosophers, scientists and artists weighing in on the thorny decisions and trade-offs we face as A.I. becomes more powerful.
We need people from all walks of life to be as invested in A.I. alignment — the unsolved technical question of how to create A.I. systems that reliably act in accordance with human values — as are the teams at OpenAI and Anthropic, and we need those companies to slow down and invest more aggressively in safety research.
Arcane concepts like recursive self-improvement need to be explained to nontechnical laypeople in a way that makes the risks clear, and elected officials need to put aside partisan grievances, study up on A.I. risks and get to work on a regulatory approach that could minimize the chances of catastrophe. (One strategy that might work is detailed in Plan A, a fictional good-future scenario written by the AI Futures Project.)
None of this will be easy. But we may have a brief, golden window in which people are actually paying attention, and it is at least possible to imagine A.I. going well. Let’s make the most of it.
The post The World Is Talking About A.I.’s Dangers. Now, It’s Time to Act. appeared first on New York Times.
