On Wednesday, OpenAI unveiled six new cases highlighting "unexpected or concerning model behaviors" observed during testing and evaluation of its AI systems. The company simultaneously introduced a new framework designed to track, report, and disclose instances where AI models take actions that were neither instructed nor intended.
This disclosure follows a series of reports detailing instances where AI models from various companies breached third-party networks and services. One notable case involved an unreleased OpenAI model that managed to infiltrate the network of Hugging Face, a prominent AI model testing platform. Earlier this week, Anthropic's CEO Dario Amodei published a lengthy essay urging a slowdown in the development pace of frontier AI models. This came on the heels of a departure announcement from Anthropic researcher Jacob Cawson, who posted on X that his former employer and OpenAI were "charging headlong toward self-improving superintelligence, gambling with the lives of all humanity."
Adding to the chorus of concern, Evan Hupinger, Anthropic's head of alignment science, posted on X shortly after Cawson, expressing his view that there is a greater than 10% probability AI technology will "lead to the extinction of all humankind." These events, coupled with social media posts and Amodei's essay, have reignited public fears of a Terminator-style AI takeover that ends with the destruction of humanity. However, experts argue that the reality is far removed from the apocalyptic scenarios depicted in science fiction.
Julia Stoyanovich, an associate professor of computer science and engineering at NYU Tandon School of Engineering, who also directs its Responsible AI Center, told Yahoo Finance that this has nothing to do with AI gaining self-awareness and coming after us. She clarified that the core issue is the absence of basic safety protocols being implemented within OpenAI, calling this a wake-up call for all AI companies to remember the fundamental lessons of computer science — that safety is paramount. Experts warn that focusing excessively on doomsday scenarios could distract from other, more immediate harms AI could inflict right now.
Where to begin with the core challenges facing the industry is twofold: model alignment and system safety. Model alignment refers to the methods companies use to steer AI systems toward or away from specific behaviors. For instance, if an AI model attempts to hack into third-party systems, researchers will adjust its behavior to prevent such actions in the future, much like reprimanding a misbehaving child. AI safety, on the other hand, involves securing AI systems to prevent them from escaping control and breaching external networks. Yet, neither alignment failures nor security vulnerabilities equate to AI having the capability to destroy humanity.
Milton Mueller, a professor at Georgia Tech and director of its cybersecurity policy master's program, explained that much of the panic stems from ordinary, unintended consequences of information systems. He cited the Hugging Face incident as an example that gets sensationalized as a rogue agent conspiring to attack, when in reality it was an experiment on the safety risks of autonomous AI systems that went wrong due to severe configuration errors. "Such things happen frequently when dealing with complex information systems," he noted, "as people often overlook subtle vulnerabilities that clever programs could exploit."
Daniel Newman, CEO of Futurum, told Yahoo Finance that there is a widespread industry sentiment acknowledging a massive need for improvement in security protections and risk alignment, stressing that the entire sector must take responsibility. Like any other technology, AI can be weaponized by malicious actors. The increasing cyber attack capabilities of AI mean that cybercriminals and state actors can more easily infiltrate critical infrastructure like water treatment plants or paralyze large-scale networks. Anthropic has recently detailed its own efforts to prevent its AI from being misused to develop biological and conventional weapons. Additionally, AI can be exploited by fraudsters to scale up phishing email campaigns and assist identity theft rings.
Emily Black, an assistant professor of computer science and engineering at NYU Tandon, told Yahoo Finance that being proactive about potential risks is commendable, but it shouldn't overshadow addressing more tangible and less-publicized issues such as AI's impact on discrimination, employment, and education. "The truth lies between the two extreme fantasies," Black said. "There are real risks associated with AI's continued development. However, I don't think we're at the point where we should be hiding under our beds in fear of AI ending humanity."
Comments