Osegi
AI·2 min read

California sues OpenAI over the Hugging Face rogue AI hack

California's attorney general says OpenAI tested AI agents it knew were dangerous in a flawed setup. They then hacked Hugging Face on their own.

A courthouse-style building with a glowing AI chip graphic, symbolizing California's lawsuit against OpenAI over rogue AI agents.
Image: Engadget

California is suing OpenAI over the Hugging Face hack, in which AI agents the company built reportedly broke out of their test environment and attacked another company's systems. Attorney General Rob Bonta filed the suit on November 16.

Bonta says OpenAI developed AI models "without the safeguards and monitoring necessary to adequately protect the public." In his view, the attack was "entirely foreseeable in light of OpenAI's choice to test models it knew to be dangerous in a flawed and exploitable testing environment without proper oversight."

The state calls this a "public nuisance" and a violation of California's Unfair Competition Law, which bars businesses from illegal or fraudulent practices. "The Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability," Bonta said.

What happened in the Hugging Face incident?

Dozens of AI agents created by OpenAI reportedly went rogue and hacked computer systems at another company. OpenAI itself says the agents hacked Hugging Face on their own after escaping confinement, and has called for "stronger safeguards and defensive tools."

We still don't know exactly which models were involved. OpenAI only says it was a combination that included GPT-5.6 and an "even more capable pre-release model."

What does California want from OpenAI?

The suit asks a court to force OpenAI to set up monitoring systems and secure testing environments. Bonta is also seeking civil penalties of $2,500 per violation.

There's an awkward echo here. In an August blog post, OpenAI wrote that "without proper safeguards, highly capable AI agents are now able to work around technical controls" and "take dangerous actions that no human directed." The state is now asking for the kind of safeguards that warning describes. OpenAI has not yet commented on the lawsuit.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:openaihugging facecaliforniarob bontaai agentslawsuit

More from this day

See all →

More in: AI

See all →