Anthropic has granted external evaluators "employee-like access" to assess the risks of its AI models, following through on a pledge to tighten safeguards as concern grows that increasingly capable systems could spiral out of control.
The nonprofit METR, which has worked with OpenAI, Anthropic, Google and Meta, now has a team embedded in the Claude maker's office with access to its hardware and internal systems. The company announced the move on Thursday.
What METR gets inside Anthropic
Anthropic has onboarded six METR employees, providing them with company hardware, office space and access to internal tools and systems roughly on par with what its own risk assessment teams use. Staff have been encouraged to share information with the evaluators as they would with a colleague working on risk.
The arrangement costs Anthropic nothing: it is not paying METR for the access. METR does not accept donations from AI companies or their staff, and is funded by individuals, philanthropic organizations and government bodies such as the UK's AI Security Institute.
Publishing rights and limits
METR can publish its analysis, and Anthropic cannot veto publication. The company can, however, require redactions for certain categories, including customer data, details covered by attorney-client privilege or information about model architectures.
"As models have become more powerful, they've required stronger safeguards and more challenging risk assessment," Anthropic said. "We want to be publicly accountable for the decisions we're making. But we can't publicly share all the information relevant to our safety work."
Criticism and the bigger picture
Critics of embedding evaluators argue the arrangements could create conflicts of interest, pointing to problems in the accounting sector, and say stronger government regulation is needed. Anthropic compared its approach to bank supervision, where regulators station examiners inside the largest banks.
