OpenAI and Anthropic have been calling for AI-risk watchdogs, and they just got their response.
The AI Evaluator Forum, which includes several watchdog groups, released a letter on Friday outlining “minimum conditions” that they believe labs like Anthropic and OpenAI should meet to work more effectively with those evaluating the risks of their technology.
Dozens of people from the AI ecosystem signed onto the letter, including the computer science pioneers Geoffrey Hinton, known as “the Godfather of AI,” and Stuart Russell.
This letter comes in response to Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, who both committed to welcoming third-party evaluators into their companies to assess AI risk and give them employee-level access to information amid a wave of AI-related fear online this week.
“To be credible, embedded third-party evaluators must have scientific objectivity, transparency, independence, and robust protections against interference from the evaluated companies,” the letter said.
The Friday letter says that evaluators should be allowed to be transparent and have “unfiltered communication with the companies’ boards and other privileged oversight bodies.” They could also release findings publicly.
The group suggests that these evaluators be shielded from retaliation from the AI labs and granted access to “the same relevant systems, data, tools, and physical spaces” as company assessors would.
The letter also says that each lab should bring in “differing viewpoints and areas of expertise,” including on different types of risk. Those could include risks in biological sciences and loss of control.
The forum behind the letter includes METR, a nonprofit AI evaluation group that recently investigated OpenAI’s security incident with Hugging Face, and the AI Verification and Evaluation Research Institute (AVERI), an organization led by former OpenAI employee Miles Brundage and other groups.
Other tech leaders also weighed in on the debate earlier this week. Microsoft CEO Satya Nadella said his firm welcomes the “embedded evaluators” idea.
Have a tip? Contact this reporter via email at [email protected], or over text, Signal, Telegram, or WhatsApp at 415-757-8198. Use a personal email address, a nonwork WiFi network, and a nonwork device; here’s our guide to sharing information securely.
Read the original article on Business Insider
The post The ‘Godfather of AI’ backs a new watchdog plan to track OpenAI and Anthropic’s AI risks from the inside appeared first on Business Insider.
