Anthropic is giving a team of six independent evaluators company hardware, office space, and access to internal tools so they can more thoroughly vet the AI giant’s unreleased models and agents.
In a blog post on Thursday, the Claude maker said it chose six evaluators from METR, the research group perhaps best known for its report on OpenAI’s Hugging Face agent breach, to receive “employee-like access” at Anthropic.
“We want to be publicly accountable for the decisions we’re making,” Anthropic said in its blog post. “We aim to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands.”
The choice to set the evaluators up at Anthropic fulfills a promise first made by CEO Dario Amodei last month, when he published a lengthy essay calling on frontier AI companies to slow down in the wake of the Hugging Face breach.
“Anthropic is unilaterally committing to this step now,” Amodei said at the time of embedding safety evaluators.
Reacting to Amodei’s essay, OpenAI CEO Sam Altman said his company would also embed evaluators, though that hasn’t happened yet. Anthropic’s Thursday disclosure marks the first time a frontier AI company has taken the step.
Within the last year, a steady drumbeat of incidents in which rogue AI agents break free from their guardrails and wreak havoc on the wider internet has spurred calls for action from politicians and tech CEOs alike.
Most recently, an Anthropic agent submitted an erroneous tip for an unsolved murder in Philadelphia, and another agent submitted 20 visa applications on a US government website, according to published reports.
In its blog post on Thursday, Anthropic said the evaluators have free rein to publish their findings, with a couple of caveats prohibiting the METR team from disclosing customer data, model architecture, and attorney-client privileged information.
Anthropic also said it holds the right to require METR to redact information. “METR is free to characterize our redactions publicly, and say whether it believes the redactions were justified, as long as it does so without revealing the redacted information,” Anthropic said.
The access does not extend to the wider METR team.
The six chosen researchers, whose names were not immediately published on Thursday, will be “siloed” from the wider METR team.
“Embedded evaluation gives a few qualified people working under Anthropic’s security standards a close view of how we operate,” Anthropic said. “It’s not a way for other organizations to obtain our information, or that of our customers and partners.”
METR did not immediately respond to a request for comment on Thursday.
Read the original article on Business Insider
The post Anthropic just opened its doors to outside safety watchdogs. Here’s how it will work. appeared first on Business Insider.
