We have been here before. Scientists warn that a promising technology might instead wipe out humanity. Its creators sound the alarm; others warn that slowing down risks losing the race to innovate.
It was 51 years ago, and 150 people assembled in the chapel at Asilomar, a rustic conference center on the Monterey Peninsula in California, to address the dangers of recombinant DNA, a discovery only a few years old. Scientists for the first time could combine bits of DNA from different organisms, creating new hybrids that could potentially cure diseases, but that might also unleash virulent new ones.
In that case, scientists did pause and come together to self-regulate how they could safely proceed. The group included the nation’s leading biologists, but also lawyers. The rules that they created did not shut down innovation. Instead, they created the biotechnology industry as we know it.
Asilomar, as the gathering became known, took on such a mythic status in the relationship of science to society that leading A.I. scientists gathered in the same place in 2017 to try to set guardrails for the development of their new models.
That meeting produced vague pledges, but now A.I. leaders are again sounding the alarm, facing liability and public fears, and asking again for regulation. And the controversies of 50 years ago — not only the one at Asilomar but also those that came after — still offer lessons.
“There are so many echoes,” said Luis Campos, a historian of science at Rice who organized a recent 50th anniversary symposium on the recombinant DNA controversy in Cambridge, Mass., where participants drew parallels with artificial intelligence. “It’s almost as if the same exact words or phrases are being used.”
In the 1970s, the new DNA technology offered the promise of creating new medicines, and producing old ones like insulin and antibiotics faster and less expensively.
But it also allowed genes to be manipulated in ways scientists could not fathom and possibly could not control. The first recombinant included DNA from a virus known to cause cancer; another had been programmed to resist antibiotics. The risk of contagion, one horrified scientist warned, put humanity on the brink of another Hiroshima.
In two urgent letters in 1973 and 1974, scientists first called for “prudence” — the same word used this month by Anthropic’s Dario Amodei in his 3,800-word plea for regulation of A.I. Then they called for a moratorium on all recombinant DNA work until it could be done safely.
Work stopped until scientists gathered at Asilomar aiming to come up with regulations to end the moratorium. Organizers warned the group, which included scientists from other countries including the Soviet Union, that if they couldn’t self-regulate, Congress would do it for them. They wanted to avoid legislation “at all costs,” one scientist later recalled, because lawmakers might get it wrong and it would be harder to change.
They debated the potential hazards of the work over four heated days. On the final evening, the lawyers in the room steered the discussion to potential courtroom scenes. That prompted scientists to propose specific regulations around physical containment, scaled from P1 to P4 according to perceived risk.
These new rules, which the National Institutes of Health later approved and enforced, included negative-airflow rooms and pressurized protective suits for labs doing the riskiest work. Microbes had to be programmed to die outside lab conditions, and scientists banned the creation of deadly toxins or antibiotic-resistant bacteria.
Some scientists at Asilomar cautioned that the new regulations would only incite public alarm. They did.
Local officials warned of a real-life “Andromeda Strain,” a movie that had recently appeared in theaters across the country, in which an uncontrollable mutant microbe obliterated an entire town.
The most colorful and consequential controversy was in Cambridge, Mass., when Harvard applied for a permit to remodel a lab to do P3-level research. The city’s mayor, Alfred Vellucci, was a sixth-grade dropout fighting a long grudge match against the university. (He once passed a city ordinance that briefly renamed Harvard Square for Christopher Columbus.)
Mayor Vellucci threatened a citywide ban on recombinant DNA experiments and hauled prominent scientists before the City Council, demanding to know whether they were dumping deadly contagions into the sewers and why the work could not be confined to the desert, like the Manhattan Project.
He instructed the scientists to explain the risks of recombinant DNA in plain language the public could understand: “We don’t understand your alphabet.”
The hearings resulted in a three-month moratorium and a new nine-member citizen board to decide whether the city should allow the work. That group, including a nun and a nurse, toured labs at the Massachusetts Institute of Technology to view the safety standards set up after Asilomar and conducted a public mock trial on DNA, with biologists on either side as witnesses. Scientists set up booths at street fairs to explain their work to the public.
The result was the country’s first municipal biosafety ordinance, which looked much like the Asilomar guidelines, except that it banned the riskiest lab work, and had involved the public.
Now, public trust in A.I. companies has plummeted, with worries that jobs could be lost to their models or chatbots could lead users to self-harm. Some experts are warning that a relentlessly self-improving model might set off a cyberattack or biological warfare. Anger over data centers straining water and electricity supplies has led to municipal bans and politicians losing their seats.
One major difference between then and now was that an industry had yet to form around the new technology. There were no biotech companies before Asilomar. The scientists who worked on recombinant DNA did so in academic labs and understood they were dependent on federal funding.
“No one was making the argument in the 1980s that the American economy was being kept afloat because of laboratory construction,” said Robin Scheffler, a historian of biomedical science at M.I.T. whose research has documented the Cambridge controversy.
Still, the biotech industry began flowering almost immediately after Asilomar and then boomed, transforming Cambridge into the nation’s biotech hub because the new regulations actually provided stability and predictability. (By 1982 Mayor Vellucci declared at a ribbon cutting for Biogen that he had no fear of recombinant DNA as long as it paid taxes.)
Biotech still commands a larger market share than the A.I. industry. That might reassure those who argue against a slowdown, as President Trump and some A.I. leaders have.
And while lawyers at Asilomar only imagined the threat of liability, for A.I. companies the threat is quite real, as illustrated in Meta’s $18 billion settlement with states that accused it of knowing the risks of its social media platforms and hiding them.
Skeptics say that is why the A.I. industry is suddenly making warnings and seeking regulation once again.
When they emerged from their Asilomar in 2017, A.I. leaders including Elon Musk and Mr. Amodei broadly urged “beneficial” use of the technology but no specific regulations. That was before the launch of ChatGPT.
“When they realized the money that was involved, it was kind of ‘never mind,’” said Sherry Turkle, an M.I.T. sociologist who has written extensively about the social implications of new technologies, including in a new book, “Artificial Intimacy.”
Six years later, many of the same signatories signed on to two letters — and as with the biologists of the 1970s, they were increasingly urgent.
In March 2023, Elon Musk again and Steve Wozniak of Apple, among others, called for a six-month pause to study the risks of anything more powerful than GPT-4. In May of that year, the leaders of OpenAI, Google’s DeepMind and Mr. Amodei warned that A.I. posed an existential threat and should be planned for in the same way nations prepare against pandemics or nuclear war.
The government did not make moves to seriously rein in A.I. The companies kept moving ahead. Anthropic released Claude to the public that July.
“There’s little good will to think that anything other than profound fear about liability leads them to come back to us again and say, ‘Now it’s really dangerous,’” Dr. Turkle said.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.)
As with recombinant DNA, a big question is how to contain the risks. But with A.I., it’s less clear how, or who would enforce the new rules. A.I. cannot be contained by protective suits and sealed rooms.
“You have to have the accountability barriers, too,” Dr. Campos, at Rice, said. “Who is accountable when something goes wrong? Who are the other people who have the right to access information to see what happens and what went wrong?”
“What mattered at Asilomar was the lawyers who said: You need to be thinking about these larger consequences, and you are not the only ones to make this decision,” he said. “It’s not just going to the pretty place on the California coast and let the experts decide.”
The post How Scientists Contained a Threat That Could Have Destroyed Humanity appeared first on New York Times.
