US and Chinese AI researchers are calling for global cooperation on the tech, fearing that its unchecked development could lead to an unimaginable catastrophe.
“AI is a global technology with global benefits, global harms, and a consistent tendency for new capabilities to eventually proliferate,” Stephen Casper, a computer scientist at MIT who spoke at a major AI conference in Beijing this month, told Wired.
“One thing that almost everyone in AI can agree on right now is that AI doesn’t need a Chernobyl moment,” he added.
Casper didn’t elaborate further on this analogy, but by invoking the infamous nuclear disaster, it’s clear that the fear isn’t just over the catastrophe itself. It’s also that it could irreversibly tarnish the public perception of AI and stifle its development, much like how the Chernobyl disaster continues to hang over nuclear power writ large.
Flavors of AI doomsaying vary dramatically, ranging from Skynet-style scenarios to mass unemployment. But more recently, as it’s become clear that one of AI’s most practical applications is generating code, experts have been sounding the alarm on AI’s potential to disrupt cybersecurity. Hackers could easily abuse AI agents and coding tools to orchestrate devastating cyberattacks, both increasing the scale of these attacks and lowering the skill needed to carry them out.
This narrative has in part been stoked by AI companies, as when Anthropic announced, and then refused to publicly release, its Claude Mythos model, purportedly because it was so powerful that it could easily break into “every major operating system and every major web browser.”
The risk is deepened by the rise of open-source and open-weight AI models which are free to use and are favored by researchers for the transparency they provided — but lack the oversight that leading commercial AI models do. A source at one of China’s leading AI companies told Wired that security concerns are one of the reasons that advanced models in China are no longer being released as open source.
Lin Yun, a professor at Shanghai Jiao Tong University warned that he expects hackers to gain an edge using AI in the short term, but that in the long term, AI could be also used to shore up cybersecurity, underscoring the need for global cooperation.
“If different countries understand the risks in similar ways, it becomes easier to develop shared safety principles and technical standards,” Yun told Wired. “The key is to find areas where sharing can reduce systemic risk without exposing sensitive operational details.”
Cooperation on this issue between two geopolitical rivals may seem far-fetched, especially since the US and China also happen to be the out-and-out leaders in AI. But Casper compared the current situation to how the US and the Soviet Union worked to rein in nuclear threats, despite aggrandizing their own nuclear stockpiles.
More on AI: Anthropic Scared, Calls for Global Freeze on AI Advances
The post Top AI Researchers Terrified of a “Chernobyl Moment”: a Mass Casualty Event, or Worse, That Turns the World Against AI Forever appeared first on Futurism.
