ResearchClaude

Anthropic gives METR employee-like access for safety

Original title: An update on independent evaluators

Plain language

What it is: Anthropic announced it is working with independent external organizations to evaluate the safety of its AI models. The company is giving a team from the nonprofit METR deep access to its systems to conduct these risk assessments.
How it works: Anthropic has onboarded six METR employees, providing them with company hardware, office space, and access to internal tools. These evaluators will analyze risks and publish their findings without Anthropic having veto power over the publication.
Where it is available: This partnership applies to Anthropic's frontier models and internal risk assessment systems, excluding sensitive data like customer information.
Example: The nonprofit METR will publish its independent analysis of Anthropic's risk levels, incidents, and safety practices.

What it is: Anthropic is implementing a third-party evaluation framework by granting independent organizations access to its models and internal systems. The initiative currently includes a deep partnership with METR and early-stage collaborations with SecureBio, Accenture, and Hugging Face.
How it works: The company is establishing a three-tier access model for evaluators. In the highest tier, used with METR, external researchers receive company hardware and access to internal tools on par with Anthropic's own alignment teams to evaluate risks.
Where it is available: The access is granted within Anthropic's internal environment, excluding specific categories such as customer data, attorney-client privileged information, and model architectures.
Example: SecureBio is using its access to review the chemical and biological weapons chapter of Anthropic's August Risk Report.

The headline and explanation on this page are based on Anthropic's post. The full details are in the original.

An update on independent evaluators