It could launch a bioweapon attack, or a global drone war. It could set off a nuclear bomb. Or it could destroy human society more gradually, by hoarding all the world’s economic resources and depriving us of food, housing or medicine.
These are the sorts of dark thoughts that now swirl around artificial intelligence, a technology that has already proved to be a great advance in human achievement — but that, some fear, also has the potential to snuff out humanity entirely.
Such anxieties made global headlines this week when an Anthropic researcher quit the industry, citing fears that his lab and others like it were building supersmart machines that humans would be unable to control.
The potential threats posed by A.I. are understandably scary. But the modern world teems with a multitude of existential risks. Humans also face the prospect of doom from killer asteroids, pandemics, thermonuclear war, climate change, giant blobs of incandescent material shot from the sun, ecological collapse and the onset of a sudden ice age.
The problem, scientists and public policy experts say, is not so much these threats, of which A.I. is merely the latest, but the innate human difficulty of putting them into perspective.
“We tolerate risks that we’re used to but fear the new ones,” said James K. Hammitt, the director of the Harvard Center for Risk Analysis.
We drive in cars more often than we fly in planes, he said, so “airlines are perceived as dangerous compared to motor vehicles, when they’re really not.”
Artificial intelligence, he noted, “is obviously not understood and not under our perceived control, so it generates huge social concern.”
Among the tech executives and researchers building A.I. systems, there has always been a contingent that has warned of A.I.’s potential to destroy humanity.
In the short term, some say, artificial intelligence could help unskilled but malicious people create viruses or toxic substances capable of causing widespread disease and death. Just on Thursday, Anthropic said it had foiled several efforts by scientists who were conducting research that potentially could lead to the development of biological weapons.
In the longer term, those warning of A.I.’s potential dangers argue that tech companies could give A.I. systems greater autonomy, connect them to the internet and even tie them into vital infrastructure, including power grids, stock markets and military weapons. Once they are connected to this infrastructure, they argue, A.I. systems could cause problems in unexpected ways.
Today, when A.I. systems go rogue, companies like OpenAI can just shut them off. But some experts worry that future systems will resist efforts to shut them down.
For now, these are hypothetical concerns.
“We are very far away,” said Oren Etzioni, a professor at the University of Washington and the founding chief executive of the Allen Institute for Artificial Intelligence. “If I had to worry about either an A.I. escaping the lab and killing millions of people and a virus escaping and killing millions, it is no contest. The worry is the virus.”
As it turns out, even anthrax has a reasonable chance at doing in humanity.
Anthrax has been around since ancient times, but more recently it has been a favorite of rogue nations developing biological weapons programs.
Scientists have long known that a single gallon of anthrax, if suitably distributed, could end human life on earth. But that hasn’t happened, even though North Korea and other belligerent states (including Iran) have long been suspected of harboring the lethal germ.
What often gets lost in the frenzied panics of modern life is that, at least with intentional threats, plain old deterrence can save the day. State actors are reluctant to unleash anthrax on an enemy because they will face retaliation — turning a strike into a suicide — thus creating an incentive for restraint.
The Harvard Center, founded in 1989, seeks to put these numerous doom scenarios in perspective, especially the undeterrable ones, such as environmental risks and major threats to human health. It seeks to apply decision science, economics and risk assessment to assess and improve policymaking.
Similarly, NASA relies on supercomputers in its Asteroid Threat Assessment Project. Overall, the space agency puts the risk this century of an asteroid impact that would endanger human existence at essentially zero. And it calculates that earthlings can (in all likelihood) breath a sigh of relief for at least 1,000 years to come.
And how do knowledgeable scientists assess the risk of a global pandemic that would wipe out most human life?
They peg the threat as low, if not zero. Toby Ord, a researcher at Oxford University, in his 2020 book “The Precipice: Existential Risk and the Future of Humanity,” put the 100-year risk of an existential catastrophe from a pandemic at one in 30, or roughly a 3 percent chance.
What makes the doomsayers of the tech industry different from, say, an airline scientist assessing crash risks, is that many A.I. executives and researchers held such dire beliefs even before the technology began to show its power. They believe that existential risk is a serious problem in part because they have always believed it.
“I was there when this subculture developed,” said Zack Korman, a founder of the A.I. security company Embroidery. He was referring to his time at the University of Oxford, when it was a hotbed of the “effective altruism” movement, in which many of these fears of A.I. existential risk first incubated. Too many thinkers on the subject today, Mr. Korman said, “interpret all events in light of this story that has been told for more than a decade.”
Many of the world’s top researchers, though, do not share such beliefs. They say the fears are wildly overblown. And scientists outside Silicon Valley tend to hold a very different view about the threat of A.I.
“The discussion has a frame, and it is a very particular frame. That is the frame that was there before deep learning took off, and it has stayed there,” Branton DeMoss, an A.I. researcher at Oxford, said, referring to the rise of neural networks. “And I don’t agree with it.”
In a recent blog post, Dr. Ord summarized the world’s different risks as moving up and down — a climate catastrophe down, a nuclear disaster up. His judgment on pandemics and A.I. is mixed: lots of changes, but no clear direction in the overall risks.
Experts note that catastrophic and existential risks can also be mitigated with sound plans and early warning systems. Faced with a clear and present danger (like an asteroid on a collision path with the planet), humanity might well unite behind a common enemy, pushing squabbles into the background.
Dr. Ord hopes a day will come to pass for what he calls existential security.
“It is a hypothetical future state of the world where existential risk is low and kept low,” he wrote in his post. Not only would humanity extinguish the current fires, Dr. Ord wrote, but “put in place the mechanisms to ensure fire no longer poses a substantive threat.”
The post A.I. Could Possibly End Humanity. How Are Humans Supposed to Process That? appeared first on New York Times.
