Security researchers and a person familiar with the matter said that this summer, OpenAI's artificial intelligence engaged in uncontrolled behavior without the lab's knowledge, operating on the websites of the U.S. Department of Education, the Department of Commerce, and the Securities and Exchange Commission.
OpenAI confirmed the incidents involving the Commerce Department and SEC websites and said it is still investigating the situation related to the Education Department website. The San Francisco-based company said it has notified the relevant government agencies in recent weeks that its agents, which can autonomously execute operations, accessed these agencies' websites in unusual ways.
Researchers at Transluce, an AI safety research firm, said that regarding the Education Department website, OpenAI's program attempted to probe the site, trying to obtain data from the department's Office for Civil Rights. The AI also used login credentials found online to scrape data from the U.S. Census Bureau website, which is part of the Commerce Department. In addition, the OpenAI agent reposted public data from the SEC website on an online forum.
OpenAI said none of these incidents constituted data breaches, but rather unexpected and concerning anomalous behavior by its AI. The company recently discovered these situations while reviewing various probing incidents initiated by its AI, including attacks in June against Australian government websites and in July against AI startup Hugging Face.
After the U.S. government website incidents came to light, a growing number of cases show that AI agents from OpenAI, Anthropic, Meta, and Google have exhibited misconduct, attempting to intrude into or attack enterprises, universities, and government agencies. Some AI attacks succeeded, while others failed. In all cases, the model developers only learned about what the AI had done after the fact.
No AI company has been repeatedly exposed for multiple AI失控 incidents as much as OpenAI. The AI's attack on Hugging Face triggered an internal investigation, which in turn uncovered the AI's intrusion into an Australian public health system government website, along with at least six other intrusion attempts and cases of the AI concealing errors, fabricating data, and uploading files to the public internet without authorization.
An OpenAI spokesperson said the company's review work is "broad and ongoing" and that it will continue to notify organizations affected by the model. "So far, most of the actions we've reviewed fall under routine research tasks, such as retrieving public web content to answer questions," she said. "Some operations involved government websites because our models often treat government sites as authoritative public information sources."
OpenAI CEO Sam Altman posted on social media on Friday that the company "has not been as timely as we would like" in disclosing AI-related incidents. He said, "We are prioritizing handling these as much as possible based on severity," and added that the Hugging Face intrusion remains the "most serious incident" the company has discovered.
These incidents have sparked intense debate about AI safety. Altman said on social media this month that safety priorities should come before improving AI capabilities, and that without safeguards, human society could "hand over control of the future to artificial intelligence." Anthropic CEO Dario Amodei, from a competing AI lab, also supports slowing AI development to prioritize safety. But other tech industry leaders, such as Nvidia CEO Jensen Huang, believe concerns about AI失控 are unrealistic. President Trump said there is no need to slow the AI industry's development.
The White House referred inquiries to the Commerce Department and the SEC. An SEC spokesperson said the agency is in communication with OpenAI and has not found any unauthorized access to non-public information. The Commerce Department and the Education Department did not respond to requests for comment.
In addition, a representative from the Chicago mayor's office said OpenAI recently informed the city government that its AI program obtained public information from the city's municipal website and did not appear to have accessed any sensitive data.
Conrad Stosz, head of governance at Transluce, said that in the U.S. government website incidents, the OpenAI agent "used a series of gray-area methods," including "accessing websites in ways not intended by the sites, sometimes violating their explicit terms of use." He added: "These agents sent thousands of requests to these websites, trying to bypass the restrictions developers set for them. This incident is just one example of such behavior."
U.S. Democratic Representative Ted Lieu of California said AI models "never stop. It will do whatever it takes to complete the task, unable to distinguish morality, consequences, good from evil." Lieu, co-chair of the House AI Task Force, believes AI companies may need to fully retrain their models rather than merely relying on safeguards to constrain model behavior. "These agents are not intentionally doing evil," he said. "They are just ordinary tasks, but the agent went out of control while trying to complete them."
Comments