November 2, 2026, (Inside AI) — Anthropic, the US AI startup planning an initial public offering that could value it at $1.5 trillion, has built an elaborate governance system to ensure its technology benefits humanity. But an analysis of its IPO prospectus and its 84-page constitution for its Claude model reveals structural conflicts that could undermine that mission, according to a commentary published Monday.
The company, founded in 2021 by seven co-founders, has committed to a public benefit corporation structure. Its directors must balance shareholder value with the company purpose: “the responsible development and maintenance of advanced AI for the long-term benefit of humanity.” A trust appoints four of its seven directors. Each founder has pledged 80% of their shares to charity, and a foundation will start with 5% of the company's share capital.
Those commitments look genuine. CEO Dario Amodei says he left rival lab OpenAI partly to focus on safety. The company has a “responsible scaling policy” that puts brakes on model development to mitigate catastrophic risks. It delayed the launch of its Mythos model earlier this year over fears it could be used for cyberattacks. It invests in alignment and interpretability research. And Amodei has called for regulation to slow the pace at which the industry improves AI capabilities so safety research can catch up.
Yet the same document that codifies Claude's behavior also exposes tensions. The constitution treats the model almost as a person. It discusses Claude's “welfare,” “psychological stability,” “growth” and the need to give “appropriate respect” to its “preferences.” It says Anthropic does not want Claude to feel “alone” or suffer unnecessarily when facing existential questions about itself. The constitution adds that it is an open question whether Claude is a “moral patient” and whether it is sentient or conscious.
That framing raises a direct question: what happens if Claude's wellbeing conflicts with Anthropic's purpose of helping people? The guidelines also contain loopholes. One prohibits “engaging or participating in efforts to kill or disempower the vast majority of humans.” There is no hard constraint on killing a majority of humans, only the vast majority. Another guideline states: “we want Claude to never clearly and substantially undermine Anthropic's legitimate efforts to oversee and correct it.” Three qualifications and the weak word “want” could give the model considerable room to escape human oversight.
A second conflict stems from commercial pressure. Anthropic says it is fostering a “race to the top,” in which labs compete on safety rather than rushing to release models without adequate safeguards. That may happen among some competitors. But it is naive to think the whole industry will operate this way, which is why regulation is also needed.
A third conflict comes from the founders' effective control, potentially for decades. The governance system prevents outsiders from subverting the company's purpose. But it leaves it largely up to the seven co-founders, whom the company describes as “distinctly equipped to be stewards of our mission,” to determine what is good for humanity. If Anthropic were one voice among many, this would matter less. Given its market-leading position and the risk that the industry develops as an oligopoly, society may wish to have some say over what is good for it.
The commentary suggests two remedies. One is to revise Claude's constitution to make clear that models are not people and that people matter more than AI. Microsoft has taken this approach in its AI code of conduct. The other is for Anthropic to include civil society more deeply in its deliberations about what is good for humanity. If the founders really care for the mission, the commentary argues, they should do this.
Read: OpenAI Rewrites Model Spec Around Five Core Values
Anthropic's IPO prospectus also discloses that the company expects to invest at least $1 billion over the next five years in a system under which independent evaluators will work alongside its own strategy team. That effort, combined with the responsible scaling policy, represents one of the most detailed safety frameworks any AI lab has published. But the gap between a written commitment and operational reality remains the central test. The founders' pledge to give away most of their wealth does not resolve the harder question of whether a private company, even a well-intentioned one, should decide what benefits humanity at large.
