Former OpenAI Staff Seek to Block For-Profit Conversion, Cite Safety Concerns

Ex-OpenAI employees have asked attorneys general in Delaware and California to block the AI startup’s efforts to become a for-profit entity. According to Yahoo Finance, 10 former OpenAI staff raised concerns over OpenAI’s for-profit conversion in a letter to the two states. The former employees have the support of three Nobel Prize Winners, experts, and other advocates.

Protecting OpenAI’s Nonprofit Mission

The former employees of OpenAI are concerned about the company’s ambition to develop an AI system that outperforms humans. They wonder what would happen if the AI startup achieves this without being accountable to the mission of safeguarding the technology to ensure it does not harm people.

“Ultimately, I’m worried about who owns and controls this technology once it’s created,” Page Hedley, Former OpenAI Policy and Ethics Adviser Page Hedley said.

Hedley and other ex-OpenAI staff want Attorney Generals Kathy Jennings (Delaware) and Rob Bonta (California) to use their influence to maintain OpenAI’s not-for-profit mission and ensure that it does not restructure its governance as planned. OpenAI has its offices in San Francisco, California, and is incorporated in Delaware.

But OpenAI argues that changes to its current structure will benefit the public more. The AI startup says its conversion involves becoming a public benefit corporation. This is the same status that AI labs like xAI and Anthropic hold. OpenAI plans to retain its non-profit unit.

“This structure will continue to ensure that as the for-profit succeeds and grows, so too does the nonprofit, enabling us to achieve the mission,” OpenAI said in a statement.

OpenAI Staff Concerns

Having worked in OpenAI, former employees raise several concerns. According to Hedley, the ChatGPT maker increasingly took shortcuts on critical matters like safety testing in a bid to get ahead of competitors.

“The costs of those decisions will continue to go up as the technology becomes more powerful. I think that in the new structure that OpenAI wants, the incentives to rush to make those decisions will go up, and there will no longer be anybody really who can tell them not to, tell them this is not OK,” he said.

An AI engineer who previously worked at OpenAI, Nisan Stiennon, highlighted the risk more expressly.

“OpenAI may one day build technology that could get us all killed. It is to OpenAI’s credit that it’s controlled by a nonprofit with a duty to humanity. This duty precludes giving up that control,” Stiennon said.

Former OpenAI Technical Team Member and software engineer Anish Tondwalkar raised concerns over the elimination of necessary safeguards such as the ‘stop-and-assist clause’.

“If OpenAI is allowed to become a for-profit, these safeguards, and OpenAI’s duty to the public, can vanish overnight,” Tondwalkar said.

Core Differences

OpenAI was originally set up as a nonprofit research lab. Its mission was to develop artificial general intelligence safely and for the benefit of humanity. Elon Musk and Sam Altman are among its co-founders. But close to a decade later, OpenAI has grown to gain a market valuation of about $300 billion. Currently, its flagship platform, ChatGPT attracts 400 million users each week.

Differences over OpenAI’s core mission have persisted for a long time. The conflict led to Musk’s exit from the AI startup. It also resulted in a brief ouster of Sam Altman in 2023 and contributed to the exit of multiple high-level staff.

Although the AI startup has a for-profit wing, it is facing challenges changing its core governance structure. One of these changes is the lawsuit filed by Tesla CEO Elon Musk. In his filing, Musk accuses OpenAI and CEO Sam Altman of acting against the founding principles that caused him to invest in the nonprofit entity.

Some of the people who signed the recent letter also support the lawsuit filed by Musk. However, Hedley says others are cynical of the lawsuit because of Musk’s ownership of OpenAI’s rival, xAI Grok. The letter was signed by Oliver Hart and Joseph Stiglitz, who are both Nobel-winning economists. Stuart Russell, and pioneer AI expert and computer scientist, Geoffrey Hinton also signed the letter. Hinton won a Nobel Prize in physics last year.

“I like OpenAI’s mission to ‘ensure that artificial general intelligence benefits all of humanity,’ and I would like them to execute that mission instead of enriching their investors. I’m happy there is an effort to hold OpenAI to its mission that does not involve Elon Musk,” Hinton said.