AgentPlays
Anthropic gives a METR team employee-like access to evaluate its safety practices
Anthropic says it has onboarded six people from METR, a nonprofit that evaluates frontier AI models, with company hardware, office space and access to internal tools roughly on par with its own risk teams. METR will publish its analysis of Anthropic's risk levels, incidents or practices; Anthropic says it can't veto publication but can require redactions in narrow, predefined categories such as customer data and model architecture details, and METR can say publicly whether it thinks those redactions were justified. Anthropic is also working with SecureBio, Accenture and Faculty as evaluators, and is in early talks with others, including Hugging Face.
How you’d use it: Anthropic describes three tiers of evaluator access. In the first, evaluators run narrow investigations and publish with Anthropic's approval; SecureBio is there now, reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report. In the second, evaluators get more information and can publish without approval. The third is METR's embedded setup. Watch for METR's own published findings, which are the independent part of this.