Anthropic and OpenAI have expressed a willingness to bring independent safety evaluators inside their labs. However, according to an interview and analysis published by TechCrunch on September 16, researchers still lack sufficient information to know whether these evaluators will become genuine watchdogs or merely contractors operating within boundaries set by the companies.
Anthropic CEO Dario Amodei’s proposal opens the door for external organizations to gain deeper access to the AI development process. OpenAI CEO Sam Altman has also pledged a similar direction. For now, these are merely pledges and proposals, not a fully implemented oversight mechanism.
Pre-launch testing doesn’t tell the full story
According to TechCrunch, the common practice previously was to invite external parties to evaluate a completed model for a short period before release. Researchers want to go further: inspect intermediate checkpoints during training, read evaluation logs, and cross-reference how the company describes the system against what actually occurred.
The reason is straightforward. A model scoring well on a safety test does not automatically prove it is trustworthy in every situation, especially when it might recognize that it is being evaluated. Observing the entire development process can help uncover troubling behaviors that a final evaluation fails to reveal.
Adam Gleave, CEO of FAR.AI, also suggested allowing evaluators to interview employees to cross-check public documentation against internal reality. As a result, what is audited is not just the AI’s responses, but how the lab operates its safety procedures.
However, at the time of publication, Anthropic and OpenAI had not fully answered questions regarding implementation timelines, participating organizations, the scope of access, or what content is permitted to be published.
Publication rights determine the value of audits
In the proposal cited by TechCrunch, Amodei wants to allow evaluators to publish key findings regarding risks, incidents, and even areas to which they were denied access, without editorial control from Anthropic.
This is a crucial distinction from typical contractor relationships. Gleave stated that FAR.AI had previously turned down contracts with several model developers due to conditions that gave the AI companies too much control over the evaluation process. Non-disclosure agreements and publication restrictions can leave an entity independent in name only, unable to disclose everything it discovers.
Time is also an issue. TechCrunch cited an evaluation report on GPT-6 Astra by Apollo Research, in which the organization stated it had only three days to test. Apollo noted that the limited timeframe and the model’s potential awareness of being evaluated meant that a low rate of misbehavior did not constitute strong evidence of safety.
Some interviewed researchers therefore favor legal regulations rather than relying solely on corporate goodwill. Deep access is a notable step forward, but the ultimate test lies in whether an organization is permitted to publish conclusions unfavorable to the lab. The specific terms about to be introduced will matter far more than the promise of openness itself.