Anthropic gives outside AI watchdog “employee-like” access

Anthropic has granted external evaluators “employee-like access” to assess the risks of its models, following through on a commitment to increase safeguards over concerns the technology could run catastrophically out of control.

METR, a nonprofit that has worked with leading labs including OpenAI, Anthropic, Google and Meta, now has a team embedded in the Claude maker’s office with access to its hardware and internal systems.

A greater role for third-party evaluators is among the main concrete steps the labs have proposed in response to rising concern about the dangers more capable AI systems could pose to the economy, cyber security and humanity at large.

Anthropic chief executive Dario Amodei achieved a rare moment of unity with arch-rivals Sam Altman and Elon Musk in September when the OpenAI and SpaceX bosses rallied behind his proposal to embed external auditors in the leading labs.

Thursday’s announcement marks the first step to implementing these pledges. OpenAI has yet to lay out plans to bring in evaluators after Altman said Amodei’s proposals were a good idea.

“As models have become more powerful, they’ve required stronger safeguards and more challenging risk assessment,” Anthropic said on Thursday. “We want to be publicly accountable for the decisions we’re making. But we can’t publicly share all the information relevant to our safety work.”

Critics of Anthropic’s proposal to use evaluators have argued the arrangements could lead to conflicts of interest, pointing to issues in the accounting sector, and that greater government regulation is required.

Anthropic argued that sharing information publicly could benefit malicious actors or competitors and added that its evaluators will operate similarly to bank supervision, whereby regulators assign resident examiners to work inside the largest banks.

“If a bank’s risk-taking could threaten the public, third parties try to see that in advance,” the company said.

Anthropic is also under pressure to reassure investors that safety issues will not overshadow its initial public offering, expected this year at a more than $2tn valuation. It devoted a large part of investor presentations this week to laying out its safety practices.

METR was one of the organisations that investigated the hack of Hugging Face by rogue OpenAI models, reviewing thousands of logs and receiving company laptops.

Anthropic has now onboarded six METR employees, giving them company hardware, office space and “access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to.”

The lab said it has encouraged employees to share information with the evaluators as they would with a fellow employee working in risk.

Anthropic is not paying METR for this arrangement. The research group does not accept donations from AI companies or their staff. It is funded by donations from individuals and philanthropic organisations, as well as government bodies such as the UK’s AI Security Institute.

The evaluators will publish their analysis, and Anthropic cannot veto publication. However, it can require them to redact information that falls into certain categories, such as customer data, details covered by attorney-client privilege or information about model architectures.

Anthropic added it was still working with Accenture, which was named a partner on safety testing last month, with each committing at least $1 billion to set up new safeguards.

Another third-party tester is SecureBio, which reviews chemical and biological weapon risk for the company.

“We’re embedding evaluators voluntarily, and we believe every frontier AI company should do the same,” Anthropic said.

METR said: “We stand ready to work with all frontier AI developers to bring better evidence to the public.”

Financial Times
Financial Times

Source: arstechnica.com