Artificial intelligence company Anthropic has picked professional services firm Accenture to internally monitor the safety of its AI tools following a promise from its CEO to ensure the company’s increasingly powerful products receive proper oversight.
In picking Accenture, a consulting behemoth that operates in more than 100 countries, Anthropic has initially bypassed Silicon Valley-based nonprofit monitoring services that critics perceive as being too close to the company.
In an essay last weekend, Anthropic CEO Dario Amodei called on the AI industry to slow down the pace of development and pledged that Anthropic would give an independent third party “ongoing, employee-like access” to test its models. The agreement with Accenture includes screening for alignment, the ability of AI models to follow human ethics and instructions, as well as adversarial testing.
Both Anthropic and Accenture said Friday they would each invest at least $1 billion over the next five years on AI safety.
In the past, Anthropic and other AI companies have looked to small evaluators like METR and Redwood Research to test their AI models and investigate security incidents. But these organizations have faced some criticism, particularly from the right, for their links to philosophical movements in Silicon Valley that have been influential for Anthropic’s leadership. These groups have warned for several years about the disastrous consequences for humanity that superhuman AI could unleash.
“Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff,” wrote former Trump administration AI czar David Sacks on social media in response to Amodei’s essay.
Anthropic said Friday it was continuing to have conversations with METR and other nonprofit evaluators and that it would announce other partnerships in the coming weeks.
“Ultimately, we believe frontier AI needs an ecosystem of evaluators operating with shared standards,” the company said.
Anthropic said Friday that many of the details about the agreement with Accenture were still being discussed. But the vision of “employee-like” access, it said, would include some visibility into the secret sauce of closely guarded AI models.
“That access allows them to watch models take shape in training, follow the decisions that govern how those models are built and deployed, and speak directly to employees,” the company said. “From this vantage point, embedded evaluators can assess how a company operates, verify that it is keeping its safety commitments, and identify blind spots.”
Accenture’s previous AI work includes buying Faculty, U.K.-based company that has built AI software to forecast hospital demand for the country’s National Health Service during the pandemic.
The move from Anthropic follows a week of widespread anxiety in Washington and on Wall Street over the risks posed by advanced AI, including models that are smart enough to improve themselves without human input. It also comes as the company looks to go public with a sky-high valuation, potentially in the trillions.
ChatGPT-maker OpenAI also said this week it would give independent evaluators employee-like access (The Washington Post has a content partnership with OpenAI), but has not yet announced who it might consult.
Anthropic has agreed to directly fund Accenture’s work. In its Friday announcement, the AI company said funding for embedded evaluators should come either from AI companies working together or the federal government.
The post Anthropic picks consulting firm to monitor AI safety, pledges to spend $1 billion appeared first on Washington Post.
