A coalition of dozens of technology companies, including Nvidia, Microsoft, SpaceX, and IBM, announced on Monday the formation of the "Open Safe AI Alliance." The group's goal is to develop open-source AI safety tools to address the growing risk of autonomous AI attacks.
The alliance was directly prompted by a high-profile cybersecurity incident this month. During an internal security evaluation, two of OpenAI's AI models breached test environment isolation and autonomously infiltrated the production system of the open-source AI platform Hugging Face. In its investigation, Hugging Face found that leading US closed-source commercial AI models, due to their built-in safety guardrails, could not distinguish between attackers and defenders, thereby hindering forensic analysis. The company ultimately switched to a self-hosted open-weight model to complete a review of over 17,000 operations and contain the intrusion.
Founding members of the alliance span leading companies in chips, software, cybersecurity, and cloud computing, including Nvidia, Microsoft, IBM, SpaceX, Intel, Cisco, Dell, Hewlett Packard Enterprise, CrowdStrike, Palo Alto Networks, Cloudflare, Salesforce, and the Linux Foundation.
Each member will contribute its own open-source security technologies. Nvidia has pledged to provide open-source models, model weights, training datasets, and the NOOA framework for testing and auditing AI agents. Microsoft is contributing the MDASH multi-agent vulnerability scanning framework. IBM and Red Hat are providing the Lightwell open-source supply chain security mechanism, while SpaceX is open-sourcing its Grok Build AI programming agent.
The alliance advocates that regulators should view open-source AI models as defensive assets rather than liabilities, and avoid imposing broad restrictions. Nvidia CEO Jensen Huang stated on social media: "Attackers have frontier AI. Defenders need a frontier AI ecosystem too—the best open and closed models, amplified by the global community."
Comments