Among the growing fears about artificial intelligence, one threat has proved particularly alarming: the possibility that A.I. could be used to create a new wave of biological weapons.
In April, biosecurity experts warned that it was disturbingly easy to get advice from chatbots about making pathogens more dangerous. In June, scientists and A.I. executives signed an open letter calling for new controls on the fabrication of DNA because A.I. can now speed up the design of toxins and viruses.
In August, scientists reported that they had used A.I. to build viruses not seen in nature. And on Thursday, Anthropic announced that it had blocked several attempts to use its models by people who might have been trying to weaponize pathogens.
Dario Amodei, the chief executive of Anthropic, followed up with a call on Saturday to slow down the development of A.I., with a set of proposals to keep it safe.
It would be wise, Dr. Amodei wrote, to ban “using AI for the production of biological weapons or allowing users to do so.”
These developments may give the impression that bad actors can now unleash previously unknown plagues on the world, all thanks to A.I.
But Drew Endy, a synthetic biologist at Stanford University who coauthored a report on biosecurity last year, said that current chatbots, like Anthropic’s Claude, don’t represent a sea change in the threat posed by biological weapons.
“In terms of the net impact on our biosecurity risk right now, I find it to be modest,” he said.
That could change, he added, as scientists develop A.I. models specializing in biological research.
Already these models can generate genomes for viable viruses not found in nature. It’s conceivable that the models may eventually be able to use those new genomes to create new weapons.
A biological weapon can be a virus or another pathogen, such as bacteria or fungi, that is designed to be unleashed against armies or civilians. Or it can take the form of toxins produced by organisms, such as botulinum from bacteria or ricin from castor plants.
(Biological weapons are distinct from chemical weapons, such as nerve gas, which scientists synthesize in labs.)
As far back as the Middle Ages, armies tried using disease as a weapon. In 1346, the Mongols loaded the bodies of people who died from bubonic plague onto catapults and launched them into the city of Caffa on the Black Sea.
Five centuries later, after scientists developed the germ theory of disease, those efforts became more precise. During World War I, German spies sneaked into the United States and other countries with vials of bacteria. They used it to kill thousands of horses and mules.
The Geneva Protocol prohibited biological warfare, but the great powers built biological weapons anyway. Eventually, the United States and the Soviet Union held gigantic stocks, from anthrax bombs to balloons loaded with wheat-killing fungi.
After signing the Biological Weapons Convention in 1975, the United States destroyed its massive stockpiles but continued studying biological weapons to develop defenses. The Soviet Union secretly carried out more investigations of pathogens, such as smallpox, for a new generation of biological weapons.
All of which is to say the threat of biological weapons was very real long before OpenAI and Anthropic appeared. And yet as far as is publicly known, biological weapons have caused very few deaths since World War II, for reasons that remain unclear.
“We don’t know what has been tried and failed, what has been deterred, what was too technologically challenging,” said Dr. Thomas Inglesby, the director of the Johns Hopkins Center for Health Security.
That threat persists, Dr. Inglesby added, noting that someone trying to create biological weapons today can use powerful tools that previous generations could only dream of.
Synthesizing DNA has become fast and cheap, for example. In 2016, Canadian scientists reported that they had recreated horsepox, a virus related to smallpox that disappeared from nature in recent decades.
The scientists broke up the virus’s genome sequences into pieces, which they emailed to a DNA-fabricating company. The company sent the DNA molecules back to the scientists, who assembled them into a viral genome and inserted it into cells.
The infected cells then produced real horsepox viruses. At the time, this was hailed as a remarkable, if unnerving, feat. But advances in DNA synthesis and other technologies make such an exercise even cheaper and easier today.
Now chatbots might consult the scientific literature to answer questions that a biological weapons-maker would have. It’s a task at which A.I. can excel, Dr. Inglesby said, providing far more to bad actors than they could gather with simple web searches.
“It aims for the information to be usable,” he said of A.I. “It allows users to go back and forth infinite number of times to troubleshoot and explain in more detail, while providing just the right technical resources to back things up.”
Dr. Endy said he was glad to see that Anthropic was scanning queries for dangerous motives.
“If I take it face value, it appears that the team at Anthropic is trying to be a good citizen,” he said. “They have a new tool, and they’re trying to be responsible with it.”
The large language models that companies like Anthropic have built are trained on vast amounts of text, including the records of work by previous generations of scientists.
They may be able to retrieve that information quickly, or to find connections between previous findings that had gone overlooked. But they have no deep understanding of biology beyond what researchers have already discovered.
Other scientists, however, are building new A.I. models that can, to a certain extent, teach themselves biology. Instead of books and other texts, these models are trained on raw data about DNA and cells.
Scientists at Stanford University and the Arc Institute built one such model, called Evo2, which they trained on the DNA sequences of millions of species. Evo2 learned hidden patterns in genes, enabling the creation of new kinds of enzymes.
That led the scientists to wonder if they could make new genomes that could support life. The researchers put Evo2 through an additional round of training on genomes from one group of closely related viruses.
Known as microvirids, they are harmless to humans, only infecting the bacteria Escherichia coli. Evo2 offered up hundreds of thousands of new microvirid genomes, 16 of which gave rise to real viruses.
To safeguard Evo2, its creators have not trained it on viruses that infect humans or other related species. But in theory, such a version of Evo2 might be able to produce a variation on smallpox viruses.
Still, that would be no small task: the smallpox genome is 35 times bigger than the genomes of microvirid viruses. And even if all that effort paid off, it would only lead to a variation on a virus that was already turned into a biological weapon in the 1980s.
Still, Dr. Inglesby speculated that, without proper safeguards, programs like Evo2 could become more useful for making new biological weapons.
“With enough training data, these A.I. systems will be able to make predictions of more lethal, contagious, immune-evasive variants,” Dr. Inglesby said.
He worried that someday the models could come up with viruses that are not just variants, but are profoundly different from natural ones.
This week’s news from Anthropic raised more immediate concerns for biosecurity experts: The queries for information about influenza and other pathogens appear to have come from state-run labs, rather than lone biohackers.
“How a Ph.D. student can individually misuse a model for harm is only one small portion of the problem,” Dr. Inglesby said.
Dr. Endy worried that if governments start using A.I. for biowarfare research, they might help rekindle an arms race like the one during World War II.
“Today we live in a world where the U.S. State Department accuses North Korea and Russia of having programs, and suspects Iran and China,” he said. “And China accuses the United States and Russia. This is not something that was happening 10 years ago.”
Dr. Endy and Dr. Inglesby agreed that a lot could be done to reduce the risk of a catastrophe. Nations have negotiated biological weapons agreements in the past, and they can try to do so now.
The Biden administration put rules into place for DNA-synthesizing companies to screen orders for suspicious sequences. The Trump administration suspended them last year and has yet to issue new standards.
Dr. Endy argued that we have to do more than try to stop a biological weapon from being used. “We have to leapfrog ahead of that, and secure biology,” he said.
That means finding ways to defend ourselves from epidemics of all sorts, whether they’re engineered by humans or have jumped from animals into our species. Public spaces like hospitals and schools, he said, ought to be equipped with sensors that can detect airborne threats, for example.
“If I see an airborne virus in the space I’m in, I can act accordingly,” Dr. Endy said. “These biological risks are not magical.”
The post A.I. Queries From Abroad Raise Fears of a Biological Weapons Race appeared first on New York Times.
