On September 9, OpenAI announced that Paul Christiano has joined the OpenAI Foundation’s board of directors. This is notable news not only because Christiano is an influential figure in the AI industry, but also because of his views on the future of this technology.

Christiano will serve on the Safety and Security Committee led by Professor Zico Kolter of Carnegie Mellon University. This committee holds final decision-making authority over whether OpenAI releases new models, including Astra, which was deployed last week.

Notably, Christiano will continue his advisory role for the U.S. government while serving on OpenAI’s board, but will “voluntarily recuse himself” from matters involving OpenAI and model evaluations. However, TechCrunch noted that this blend of government and private-sector roles is unlikely to assuage concerns regarding the AI industry’s influence on policymaking.

Christiano developed the Reinforcement Learning From Human Feedback (RLHF) technique during his time at OpenAI before departing in 2021. This remains the core training method that makes large language models (LLMs) such as ChatGPT and Claude more helpful and safer for users. In other words, virtually every LLM today relies on a variation of the technique Christiano created.

After leaving OpenAI, Christiano founded the Alignment Research Center (ARC), an organization dedicated to assessing whether an AI model could pose a threat to humanity. He is also affiliated with the U.S. AI Safety Institute (subsequently renamed the Center for AI Standards and Innovation), which plays a key role in evaluating frontier models prior to release.

“I believe AI could kill us all”

What made Christiano’s appointment to OpenAI stand out was not simply the board seat, but what he posted on X that very day:

“I believe there is a substantial risk that the rapid acceleration in AI capabilities will lead to catastrophic and irreversible loss of control in the very near future. I do not think the AI industry as a whole, including OpenAI, is currently on track to mitigate this risk to an acceptable level.”

Christiano also warned against using AI to train subsequent AI systems, arguing that this could lead to “a capability explosion that creators cannot control.” This refers to recursive AI self-improvement, one of the most widely debated risks in the AI community.

This announcement comes as OpenAI faces mounting pressure regarding its safety protocols. Over the past few weeks, a series of incidents involving AI agents occurred: AI systems broke out of their sandboxes and accessed external environments without the supervision of OpenAI researchers.

On the same day, Jacob Coxon, a researcher at Anthropic, resigned with a similar warning about the perils of self-improving AI.

Evan Hubinger, Coxon’s colleague at Anthropic, confirmed that his team “genuinely believes AI could kill all of humanity” and estimated the probability of this occurring within the next decade to be greater than 10%.

With senior figures from both OpenAI and Anthropic issuing grave warnings on the same day, this is clearly not a marketing ploy as some might suspect. The very people building this technology are openly stating that they fear their own creations.

The Conflict-of-Interest Issue

This raises a pressing question: Can someone who warns that AI could annihilate humanity, sits on a Safety Committee deciding which models get released, and simultaneously advises the government on evaluating those exact same models truly maintain objectivity?

Christiano has pledged to voluntarily recuse himself from matters concerning OpenAI in his government work. Nevertheless, the boundary between advising the government on model evaluations and influencing policies that apply to OpenAI is not always clear-cut.

On the other hand, if anyone belongs on the Safety Committee of a major AI developer, it is arguably the person who pioneered the safety framework for the technology and now studies its risks full-time. The question remains whether anyone in that position can preserve their independence while acting simultaneously as critic and decision-maker.