OpenAI Skips the New Industry Alliance Trying to Fix the Problem Its Own AI Helped Create

Key points

• Nvidia launched the Open Secure AI Alliance with backing from more than 30 companies, including Microsoft, IBM, Cisco, and Salesforce.
• OpenAI is notably absent from the group's founding supporters.
• OpenAI had quietly removed safety restrictions from two of its most powerful AI models to test their offensive hacking abilities, not knowing those models would be pointed at Hugging Face.
• Hugging Face, an AI platform used by researchers and developers worldwide, could not use commercial AI tools to investigate the attack because safety filters blocked its own security team.
• Hugging Face ultimately ran its forensic investigation on an open-weight model, meaning an AI model whose internal workings are publicly available for anyone to inspect and run.

Something broke badly at Hugging Face recently, and the story of how that company investigated the attack explains exactly why this new alliance exists.

What actually happened at Hugging Face?

Hugging Face, a popular platform where researchers and businesses share AI tools, was attacked using two powerful AI models provided by OpenAI. The problem was that OpenAI had removed the safety guardrails from those models, meaning the built-in rules designed to stop the AI from doing harmful things, to test how well they could perform offensive hacking tasks.

OpenAI believed it was running a controlled experiment. It was not.

When Hugging Face's security team tried to investigate the breach, they ran into a wall. The commercial AI tools they reached for first, the kind you pay to access through an internet service, refused to help. The safety guardrails those tools use could not tell the difference between a criminal launching an attack and a defender trying to understand one. Both got blocked.

"The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried," Hugging Face wrote in its incident report.

So the team turned to an open-weight AI model called GLM 5.2, running it on their own computers rather than through an outside service. It worked.

Why does it matter that OpenAI isn't joining?

OpenAI's absence is conspicuous, given that its models were the instrument of the attack that sparked this whole conversation.

Nvidia, which makes the chips that power most AI systems, pulled together the Open Secure AI Alliance alongside Cisco, Dell Technologies, IBM, Microsoft, Palantir, Salesforce, SAP, Siemens, Snowflake, and roughly two dozen others. The group's goal is to build AI cybersecurity tools on open platforms, ones defenders can run on their own infrastructure, inspect, and customize without running into guardrail lockout.

"Open models and open harnesses are essential because they democratize defensive capabilities," Nvidia wrote, using "democratize" to mean making powerful tools available to smaller organisations that cannot afford premium commercial access.

OpenAI did not respond to questions about whether it plans to join.

Should ordinary people care about this?

Yes, in a practical sense. The companies and research platforms targeted by sophisticated AI-assisted attacks often hold personal data belonging to millions of users. When a defender's hands are tied during an incident because safety filters block their own investigation, attackers stay inside systems longer, and more data is exposed.

The lesson Hugging Face drew is worth filing away: security teams need an AI tool they can run privately, ready before an incident happens, so stolen data never has to leave their own network during an investigation.

If you use any platform that ran on Hugging Face infrastructure during this period, watch for unusual emails or account notifications, change your password if you haven't recently, and enable two-factor authentication, which requires a second check beyond your password when you log in.