As concerns over artificial intelligence rise, New York City lawmakers grilled representatives for some of the major A.I. companies on Monday — and emerged with little clarity over how they plan to address looming threats.
The companies did not quantify the risk of a cataclysmic event, whether they had insurance in those instances and whether they could stop an A.I. model that failed an internal safety test or external review.
In one particularly tense exchange, the City Council speaker, Julie Menin, asked each of the companies how they are quantifying “the risk imposed by A.I. in the worst case catastrophic scenario.”
“I don’t know,” said Morgan Dwyer, who runs OpenAI’s policy development and operations. “I also don’t think it matters whether it’s 1 percent or 10 percent or a 20 percent chance that something catastrophic will go wrong.”
“None of these levels is remotely acceptable,” she said.
Ms. Dwyer said OpenAI should not train models that cannot be safely kept under human control.
“I’m sorry, but to say you don’t know and it doesn’t matter is flippant at best,” Ms. Menin replied.
The daylong hearing featured testimony from representatives of four major A.I. companies — Anthropic, OpenAI, Meta and Google — who also took questions remotely. A representative for the A.I. arm of SpaceX, which is controlled by Elon Musk, did not attend the hearing despite receiving a subpoena from the Council compelling the company to attend. Ms. Menin said she would ask a judge to intervene.
Some of the A.I. companies are preparing to be publicly traded, with projections for Anthropic’s value exceeding $2 trillion in some financial reports.
The companies touted the benefits of A.I. throughout their remarks.
“I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared,” Ms. Dwyer, of OpenAI, said. “A.I. can help scientists develop new cancer treatments, help entrepreneurs build and grow businesses and help protect critical infrastructure by addressing cybersecurity threats. And in New York City, we are already seeing A.I.’s benefits.”
Less glowing appraisals came from whistle-blowers who have left A.I. companies and warned that the technology, which is rapidly evolving, poses severe risks to humanity and must be regulated.
“We do not know how to control any A.I. system yet,” Jacob Coxon, who recently quit his job as a researcher at the A.I company Anthropic, said. He attended the hearing weeks after he touched off a firestorm of concern over the grim potentials of the technology.
Proponents of the technology tout possible medical breakthroughs and freeing humans from rote, time-consuming tasks. Mr. Coxon also spoke of the promise of A.I., in particular its ability to “transform the economy.”
“But on the current path,” Mr. Coxon said, “I think it is more likely than not that humanity loses control of these A.I.s and it could end in human extinction.”
“From my experience, the companies are being extremely reckless, given the stakes,” he added. “Part of it is the culture. The companies run on a start-up mind set: move fast, break things, fix them later. That works for a photo-sharing app. It does not work for building the most powerful technology ever built.”
“None of them would quantify the risk,” she added. “How can you put on safety guardrails if you can’t even quantify, well, what exactly is the risk?”
Ms. Menin and her colleagues have introduced a raft of legislation to deal with A.I., though more meaningful control over these firms would have to come from regulation by the federal government.
One Council bill would make it illegal to market or sell an A.I. model in New York City if it has not been validated by an independent review, or if it lacks the capacity to be shut down by a human operator. Another would mandate city agencies develop an emergency response plan for A.I. threats to the city’s infrastructure and public health.
After the hearing, Ms. Menin said the companies “were not very forthcoming about the questions that we had.”
Zohran Mamdani, the New York City mayor, has called for stronger federal safeguards for A.I.
Asked Monday about the Council’s legislation, Mr. Mamdani said that he thought it was “incumbent upon all of us to play a role in responding to what is one of the most important questions of our time, which is artificial intelligence, the response to it, the regulation of it.”
He said he would review the Council’s bills, and mentioned that his administration has issued a moratorium on so-called generative A.I. for young students, while limiting it for those in high school.
The Mamdani administration also has to address another A.I.-related task; its A.I. director recently resigned. During the hearing, a representative from the New York City Office of Technology and Innovation said the office was close to filling that role. A spokesman for the agency declined to explain the departure.
The post A.I. Officials Stonewall on Questions About Technology’s Risks appeared first on New York Times.
