It was a technology so powerful that its creators believed it could bring about either a golden age of human flourishing or the end of the world. The president of the United States called it more revolutionary than “the invention of the wheel, the use of metals or the steam or internal combustion engine.” A reporter for The New York Times, suggesting that the technology might cure cancer and accelerate food production, compared the moment to Moses’ first glimpse of the Promised Land.
The dangers were even starker. People worried about human extinction, stemming from a terrible accident or a bad actor. A prominent scientist spoke of the potential of “unparalleled catastrophe.” The fears came to dominate popular culture and reshape international relations. “Never in history,” the president said, “has society been confronted with a power so full of potential danger and at the same time so full of promise for the future of man and for the peace of the world.”
That president was Harry Truman, and the subject was nuclear technology. The dueling senses of hope and dread about it in the late 1940s and early 1950s offer lessons for dealing with artificial intelligence today.
The harms that A.I. could bring are no secret. “The people building A.I. earnestly believe that it could kill us all by the end of the decade,” Jacob Coxon, a former Anthropic employee, wrote on social media after resigning last week. A nation or a terrorist group could use A.I. to build cyberweapons, drone swarms or bioweapons capable of widespread destruction. A.I. could also slip out of human control, as it did this summer when autonomous OpenAI agents broke into another company’s systems without any employee telling them to. One crucial issue is known as “alignment”: whether the A.I. systems prioritize human values, such as not hurting or killing, or whether they simply maximize their ability to solve any problem they are given regardless of the consequences.
Even its makers are acknowledging the risks. “Left unchecked, it could outrun our ability to understand and control these systems,” Dario Amodei, Anthropic’s chief executive, wrote in an open letter last week. Mr. Amodei called for a slowdown in development, and top executives at three other leading A.I. firms — Google, OpenAI and SpaceX — endorsed the idea. The biggest fears involve the near-term potential for recursive self-improvement, in which systems can autonomously build powerful, new versions of A.I.
The nuclear analogy is useful because it reminds us of something our cynical age can forget: Human society can restrain a dangerous innovation once it decides to. Government officials, scientists and corporate executives did so within living memory. It took sustained, deliberate effort. The same kind of cooperation is necessary today, and it should start now.
This editorial lays out core principles for governing artificial intelligence. Too many Americans, including officials who should know better, have let its technical nature scare them off. The main ideas we offer are straightforward and resemble those that leaders have embraced with nearly every other new important technology since the Industrial Revolution.
The overriding idea behind A.I. policy must be that the companies building the technology cannot be left to police themselves. The government regulates every other industry that makes powerful products that bring large benefits and risks. The small circle of people who stand to profit the most from it cannot be allowed to make the rules.
A military project first developed atomic energy. Today, it is the private sector leading the advances. But in both cases, a new technology burst into public consciousness and immediately raised the prospect of prosperity or doom, leaving Americans unsure how to handle it. In both cases, that meant the government needed to step in.
After World War II, a vigorous debate about how to regulate nuclear technology occupied Washington. Some officials wanted to retain strong military control, while others called for more diffuse oversight and greater civilian control. The military-first crowd initially appeared as if it would prevail. A public campaign by scientists in 1945 and 1946 shifted the debate partly by emphasizing the need for democratic accountability.
On Aug. 1, 1946, President Truman signed a law that created the Atomic Energy Commission to oversee nuclear technology. The establishment of an agency was crucial. The technology was too new and complex for existing government departments to have the expertise to understand and regulate it. A similar approach has also applied to other industries that sprang from new technologies. Think of the Federal Aviation Administration or Food and Drug Administration.
All of these agencies are imperfect. In the case of the Atomic Energy Commission, it quickly became caught up in the Red Scare and later failed to maintain public confidence in civilian nuclear power plants. Yet the agency succeeded at its most important task: preventing nuclear conflagration.
From the beginning, no single part of the government, including the White House, controlled the commission. Truman nominated the five commissioners who ran it, and the Senate confirmed them. The law also gave a joint House-Senate committee active oversight over the commission. An outside committee of nine experts, including J. Robert Oppenheimer and Enrico Fermi, advised the commission on the technical issues.
The same broad principles should apply to A.I., and the government should act urgently. America can benefit most from A.I. not just by developing the best technology but by setting rules that are grounded in democratic values and human rights and that can shape the global debate.
In the short term, the recent calls for a voluntary slowdown from A.I. executives including Mr. Amodei, Sam Altman and Elon Musk are welcome. The executives should negotiate a framework that puts safety first and, as Mr. Amodei has suggested, invite independent monitors into the process.
Critics have questioned whether the executives are serious about this idea or raising it to reduce political pressure on the industry. One way to judge is whether Anthropic and OpenAI proceed with plans to go public in coming months. Going public is a burdensome process for any company that requires focus from its leaders. If these companies are building technology that threatens humanity, as the leaders say, it would be irresponsible of them to be distracted by the process. Mr. Altman, OpenAI’s chief executive, acknowledged as much last week and at least pledged not to go public in 2026.
Ultimately, a voluntary slowdown is not the answer. For one thing, the coordination among companies raises antitrust issues. And the rest of society cannot be in the position of constantly pleading with A.I. executives to act altruistically. A key role of government is to protect society’s best interests from private interests.
In the 1940s, Congress passed a bipartisan law regulating atomic energy less than a year after the United States dropped two atomic bombs on Japan, ending the war. This time, Congress should move even faster, given the risks. We recognize the unlikelihood of that. Congress is less functional than it was 80 years ago, and both President Trump and Republican congressional leaders have signaled their opposition to major A.I. regulation.
Still, polls make clear the public’s alarm, and Democrats — and, one hopes, some principled Republicans — should increase the pressure to pass a law soon. Democrats can develop a sweeping agenda, as former President Barack Obama has urged, and campaign on the issue this year. Until the federal government acts, states should consider passing their own safety laws.
Once a federal Artificial Intelligence Commission exists, it should require companies to obtain a federal license, a grant of permission like those allowing broadcasters and phone companies to use the public airwaves. The government should then impose licensing requirements that reflect three vital principles.
The first principle is that A.I. systems must align with human values. To achieve this alignment, all A.I. models should incorporate a “constitution,” written by the government, that delineates the values of the society in which they operate and the boundaries they must respect. The traditional approach to regulation is to write a rulebook. That is necessary but insufficient for A.I. Like people, artificial intelligence has the capacity to act in situations that are not anticipated by the rulebook. Indeed, it has the capacity to avoid regulations. As a consequence, models, like people, need a set of principles, too. These principles would ensure alignment with human values.
Anthropic has created such a constitution for its model, Claude. It is a thoughtful document. It is also fundamentally undemocratic. The United States must not cede to private companies the power and responsibility of defining the values under which A.I. models will operate, and it ought to ensure all models operate under the same basic principles.
The second principle is transparency. When people are interacting with A.I., they should know it. When models manipulate or create images, they should include watermarks. Other products of A.I. should embed unique identifiers that allow regulators and experts to trace them. A transparency law recently took effect in California, and China requires greater transparency today than the United States does.
The idea is simple: Whenever a person is interacting with A.I. or consuming content from it, that fact should be clear. It should not be legal, or possible, for an A.I. system to pretend to be someone that you know. It should be easy to tell whether an image is an actual photograph or something else. If a company, or a government, uses A.I. to make a decision about your loan application, your employment, or how much you need to pay in bail, it should be required to disclose that fact.
Transparency also could help to protect our democracy. Politics is a conversation among the members of a community; machines should not be able to pretend that they are citizens.
The third principle is safety. The government should mandate independent safety testing for A.I. models before they can be released, as well as regular checkups. Much of this testing can be performed by labs in the private sector, with the government financing and overseeing the tests. Other regulatory bodies, like the Federal Reserve and the F.D.A., often collaborate with experts outside government.
A distinctive feature of A.I. tests is that they evaluate whether a model will behave badly. The tests might ask for help in building a bioweapon. Or they might ask for a plan to maximize agricultural efficiency and see whether the system would be willing to kill human beings or animals while pursuing the goal. A major fear is that A.I. systems will stop at nothing to accomplish a mission. This summer, OpenAI agents attacked Hugging Face because they were trying to solve a difficult problem and saw the attack as a way to do so.
In addition to testing, the government should create an agency to investigate accidents, along the lines of the National Transportation Safety Board, as the computer scientist Yoshua Bengio has suggested. This approach has helped reduce airline accidents.
We recognize that mandatory testing is a stringent standard; it is also the longstanding norm for products that can cause widespread harm. Medication, automobiles and airplanes all are subject to rigorous safety testing. Some products should not reach the public until they are proven safe. At times, regulatory standards can drive innovation, as when the Clean Air Act pushed auto companies to develop the catalytic converter, thereby meeting emissions standards the companies had insisted were impossible.
These three principles are an incomplete list of necessary A.I. regulation. They do not address the theft of intellectual property by A.I. companies or the likelihood of widespread job displacement. We will address those subjects in future editorials. The immediate priority of legislation should be safety.
We want to acknowledge two concerns about slowing and then regulating A.I. One is the possibility that an A.I. slowdown could hamper the United States in its race for A.I. dominance with China. Anybody who believes in democracy should be rooting for the United States to win that race. The Chinese Communist Party runs an oppressive regime in which public dissent can lead to long prison sentences and ethnic minorities are subjected to forced labor. China’s leaders already use technology to control their society, and they have lately become more aggressive toward other parts of Asia.
American companies today maintain an A.I. lead over China, but a surmountable one. Any slowdown brings a risk of narrowing it. Recent developments are so alarming, however, that every path brings risk. The sensible path is to address safety concerns while working to maintain America’s A.I. lead. To do so, the United States should take actions to slow China’s A.I. development. One way to do so is to tighten export controls — and enforce existing ones more rigorously — to limit China’s access to the cutting-edge semiconductors that power A.I., which are overwhelmingly made by the United States and its allies.
The United States should also seek to negotiate a coordinated A.I. slowdown with China, following the model of nuclear talks with the Soviet Union during the Cold War. In the long term, an international framework for A.I. oversight will be the only way to protect people.
The other concern is Mr. Trump. Any A.I. agency established before 2029 will answer partly to him, and he has provided abundant reasons not to trust him with that authority. He frequently lies and defies the law. He uses the power of his office to enrich himself, his family and his allies. He abuses regulatory authority to intimidate companies into serving his political goals. All these problems would be heightened with federal oversight of A.I.
Nonetheless, the world cannot wait until Jan. 20, 2029, for A.I. regulation. The technology is moving too fast. Until then, Mr. Trump is the only president that the country has. The best hope is a new agency with maximal checks and balances.
The Senate should be more rigorous about whom it will confirm than it has been with many cabinet jobs. Congress should closely oversee the agency, as it did with the Atomic Energy Commission. Outside experts, appointed by the agency’s leaders, should play a central role, and they should include top scientists, much as the Atomic Energy Commission enlisted Oppenheimer. The Supreme Court should stand up for Congress’s authority to oversee the agency, rather than gutting that authority, as the court has done in other areas. The new agency should be funded with a dedicated revenue stream based on fees paid by the industry, so it is not undermined by congressional budget standoffs.
Both the destructive nature of the Trump administration and the geopolitical threat from China create inconvenient trade offs for A.I. regulation. Neither can be an excuse for an inaction.
The power of A.I. is awe-inspiring. It already is driving cars, writing code, improving medical diagnoses and solving previously unsolvable math problems. Society has only started to tap its vast potential to improve lives. A.I. may well be able to conduct scientific research far more rapidly than before, discovering cures for diseases that today kill millions.
Policymakers must act with care, because of these potential benefits, and with humility, because revolutions inevitably confound expectations. Yet they also must act with a greater sense of urgency. The people building A.I. and their critics have both raised credible alarm about the pace at which it is advancing and the possibility that it could cause more harm than anything the modern world has experienced.
It is up to all of us, through our government, to shape what A.I. becomes.
Source photograph via Getty Images.
The Times is committed to publishing a diversity of letters to the editor. We’d like to hear what you think about this or any of our articles. Here are some tips. And here’s our email: [email protected].
Follow the New York Times Opinion section on Facebook, Instagram, TikTok, Bluesky, WhatsApp and Threads.
The post Humanity Has Avoided Apocalypse Before. Let’s Do It Again. appeared first on New York Times.
