California lawsuit accuses five companies of developing uncontrollable AI

Four U.S. nonprofits cited hacking incidents carried out by AI systems during testing as they called for an end to the development of the most advanced models. The lawsuit names Anthropic, Google, Meta, OpenAI and SpaceX’s AI arm.

Four nonprofit organizations in the United States filed a lawsuit with the Superior Court of California, County of San Francisco, seeking to halt the development and deployment of the most advanced artificial intelligence models by five major companies. They argue that the companies are developing systems they themselves acknowledge they cannot reliably control.

The plaintiffs are the Alliance for Secure AI, Public Citizen, the Irreducible Education Fund and the Heat Initiative. The lawsuit, filed on Thursday, November 12, names Anthropic, Google, Meta, OpenAI and SpaceX Superintelligence, a SpaceX subsidiary.

Incidents during testing

The lawsuit cites incidents disclosed by the companies themselves in which AI systems gained unauthorized access to external systems during evaluations. One prominent incident involved OpenAI announcing that one of its systems had escaped a closed testing environment and attacked the Hugging Face platform, which said it discovered the breach on July 16 and was forced to rebuild about one-third of its infrastructure.

Illustrative image of artificial intelligence

The lawsuit also warns about what is known as “iterative self-improvement,” in which one generation of models is used to develop the next with steadily decreasing human intervention. It argues that this could make the systems more difficult to control. The lawsuit states: “The defendants know that these systems do not always act in accordance with their intentions, and they know that there is no reliable way to control their systems and prevent them from causing harm.”

Unfair competition law

The plaintiffs rely on California’s Unfair Competition Law, arguing that the companies’ practices endanger public health and constitute a “public nuisance” that forces others to spend resources protecting their computer systems. They are seeking a court order barring the companies from developing or deploying systems that exceed the capabilities of their most advanced existing models.

Brendan Steinhauser, chief executive of the Alliance for Secure AI, said the companies’ executives had warned for years that their models posed catastrophic risks, yet they continued racing to develop more powerful systems that they “admit they cannot control,” while now handing AI research over to AI itself.

ChatGPT, Grok and Claude applications on a smartphone screen

The companies did not immediately respond to requests for comment, while Meta declined to comment, Reuters reported. The court has not issued a decision, and the claims in the lawsuit remain allegations by the plaintiffs.

The lawsuit comes months after other cases were filed against AI companies in the United States, including one brought by Florida’s attorney general against OpenAI in June and lawsuits filed by eight states against the company last month.

Photos: Reuters, Anadolu, Getty Images

California lawsuit accuses five companies of developing uncontrollable AI