Anthropic is breaking new ground in AI safety by giving third-party evaluators employee-equivalent access to its internal tools, hardware, and office resources. This unprecedented move aims to improve independent oversight and accountability of AI risks at a time when federal regulation remains limited.
- Anthropic grants METR evaluators deep, employee-level access to AI safety tools and information.
- METR will publish risk evaluations with limited redactions, enhancing independent transparency.
- This cooperation advances third-party oversight amid evolving US AI regulatory pressures.
What happened
Anthropic has taken an unprecedented step in AI safety by embedding six evaluators from METR, a nonprofit organization specializing in testing frontier AI models, within its teams. These evaluators have been provided with Anthropic hardware, office space, and access to internal tools and systems comparable to those used by Anthropic’s own risk assessment teams. Except for specific sensitive categories such as customer data and privileged information, Anthropic employees are encouraged to share safety-relevant information openly with these evaluators.
METR will independently analyze Anthropic's models and safety controls, publishing their findings freely. Anthropic may request redactions only to protect sensitive commercial details or security concerns. This model aims to create a direct channel of accountability by allowing third-party experts to access and evaluate critical safety information typically restricted to internal teams.
Why it matters
The move addresses a key challenge in AI development: balancing transparency with the protection of proprietary and security-sensitive information. By embedding an independent entity like METR deeply within its operations, Anthropic seeks to close the gap between external evaluators' understanding and the company’s internal knowledge of AI risks. This model enhances credibility and could become a benchmark for self-regulation in the AI industry, which has so far been largely voluntary and fragmented.
This initiative arrives amid growing scrutiny from regulators, such as the FTC subpoenaing major AI companies over consumer protection concerns. It complements ongoing legislative efforts like the bipartisan FRONTIER Act, which proposes mandatory third-party verification of AI safety. Anthropic’s approach may inform how AI companies can transparently demonstrate compliance and responsibility while managing sensitive information.
What to watch next
Stakeholders should watch METR’s forthcoming published findings closely to gauge how effective this embedded audit model is in identifying AI risks and improving safety practices. The degree of transparency and cooperation demonstrated by Anthropic could influence wider industry adoption or regulatory expectations for independent verification before model releases.
Regulatory developments are also critical to monitor. The US government’s recent shift toward mandatory disclosure of AI safety incidents signals increasing enforcement activity. How this framework interacts with voluntary cooperation efforts, such as Anthropic’s partnership with METR, will shape the evolving landscape of AI oversight and corporate accountability.