MW Will AI start 'going rogue'? The chorus of warnings is getting louder.
By Hannah Pedone
As AI models get more powerful, there's growing risk they will go off the rails or get misused by bad actors
More technology experts are warning about what could happen if AI gets abused by bad actors.
What happens if artificial intelligence truly starts developing a mind of its own?
AI researchers and executives have been raising concerns about the biological, cyber and nuclear risks of artificial intelligence for years. But now, someone at the top of one of the world's biggest AI labs is warning about what could happen if the technology starts to "go off the rails."
That would be Demis Hassabis, CEO of Google DeepMind $(GOOGL)$ $(GOOG)$.
Hassabis said in an interview this week that he worries about AI "going rogue" as the technology gets more powerful, and as agents advance and evolve into "systems that are completely capable of completing entire tasks on their own."
He worries that bad actors - be they individuals, organizations or countries - will repurpose technologies that were designed to help cure diseases or create new materials. The models could be employed for harmful ends either "inadvertently or intentionally," he said.
All frontier labs have to think about ensuring that proper guardrails are put in place so that agents simply end up doing "exactly what they've been told to do," Hassabis added.
OpenAI CEO Sam Altman this week discussed the potential for AI to help advance biology. "We're going to see a bunch of diseases get cured," he said in an interview with Axios.
However, he noted that society needs to be mindful that terrorist groups may use models to create dangerous "novel pathogens."
"That's no longer a theoretical thing," Altman said.
Anthropic CEO Dario Amodei wrote in a January essay that AI systems are inherently unpredictable and challenging to control - citing behaviors such as "sycophancy, laziness, deception, blackmail, scheming, 'cheating' by hacking software environments, and much more."
He wrote that AI companies certainly "want to train AI systems to follow human instructions (perhaps with the exception of dangerous or illegal tasks)." But Amodei added that the process of doing so is "more an art than a science, more akin to 'growing' something than 'building' it."
This week, Anthropic's limited release of a new model, Claude Mythos, brought those concerns to a head.
"Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser," Anthropic wrote of the model, while announcing that it would create a coalition of technology companies meant to help bolster cyber defenses ahead of Mythos' release.
"Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely," the company said. "The fallout - economics, public safety and national security - could be severe."
Thomas Friedman, a prominent New York Times opinion columnist, authored a piece this week titled "Anthropic's Restraint Is a Terrifying Warning Sign."
In it, he quoted Craig Mundie, the former director of research and strategy at Microsoft $(MSFT)$, who said that it's necessary to "carefully control the release of these new superintelligent models" and ensure that they "only go to the most responsible governments and companies."
Friedman wrote that he and Mundie believe that "no country in the world can solve this problem alone" and that the solution "must begin with the two AI superpowers, the U.S. and China." The two countries must learn to collaborate to "prevent bad actors from gaining access to this next level of cyber capability," he said.
Gordon Goldstein, a senior fellow at the Council on Foreign Relations who writes on AI and national security, told MarketWatch that heightened alarm in the AI community partly stems from companies like Anthropic being transparent about the risks their new models pose.
Mythos "is so powerful and potentially devastating as a cyber weapon that Anthropic will not release it publicly, which is a responsible decision," Goldstein said. But if Mythos leaks, another AI lab will likely replicate its capabilities, he noted.
Read more: Senior AI staffers keep quitting - and are issuing warnings about what's going on at their companies
Michael Vermeer, a physical scientist at Rand Corporation, said that we have not yet figured out how to align models to human values, yet we are "deploying them as highly capable agents." Still, he said there was a tendency to paint "existential threats as if they are a near certainty."
Ed Zitron, founder of a tech-focused public-relations firm and author of "The Hater's Guide To The AI Bubble," told MarketWatch that CEOs like Altman, Amodei and Hassabis are trying to "keep the mystique of the AI labs up" so they can say "you've got to give us money, because I'm the only person who can possibly fix all of this."
See also: Why the man behind 'The Hater's Guide to the AI Bubble' thinks Wall Street's hottest trade will go bust
Yet CEOs seemed to have internalized that critique in varied ways.
Hassabis acknowledged that we're in a "ferocious, commercial pressure race," not to mention a geopolitical one. He said that if he had his way, he would have kept AI in the lab for longer due to safety concerns and to focus on applications like curing cancer.
Anthropic's Amodei said in an interview in December that AI leaders, including himself, shouldn't be in charge of the future of AI, and that he's deeply uncomfortable with the idea that the policing of the technology gets undertaken by just a few companies.
When asked who elected him and Sam Altman, Amodei responded: "No one."
Meanwhile, Altman said in this week's Axios interview that "the frontier models are all sort of in the hands of pretty responsible companies."
See also: Palo Alto Networks' stock leads another dramatic software selloff. Here's what investors need to know.
-Hannah Pedone
This content was created by MarketWatch, which is operated by Dow Jones & Co. MarketWatch is published independently from Dow Jones Newswires and The Wall Street Journal.
(END) Dow Jones Newswires
April 11, 2026 07:00 ET (11:00 GMT)
Copyright (c) 2026 Dow Jones & Company, Inc.
Comments