By Katherine Blunt
Chad Olson was confused when his Gemini artificial-intelligence chatbot told him he had a family reunion planning session marked on his calendar.
Olson, who was driving home from work in Minneapolis, didn't recall scheduling the event and asked the bot to summarize some of his recent emails.
A woman named Priscilla had sent several notes asking him to pick up alcohol including Captain Morgan rum and Fireball whiskey, Gemini said. It said someone named Shirley had asked him to buy Klondike bars. Olson had no idea who those people might be.
"Seems like lots of people are reaching out to ask for different things!" Gemini said.
Perplexed, Olson asked Gemini what account it was pulling from. The email address Gemini cited wasn't his.
What appeared to Olson to be a data breach was actually a fabrication by the chatbot, Google told The Wall Street Journal. The account it referenced wasn't active, and the senders wanting liquor and ice cream didn't exist, the company determined in an internal probe. Google said Gemini hallucinates less than other models and that it is working to improve it.
As AI tools have grown vastly more powerful and sophisticated, millions of Americans have come to rely on them as everyday tools for work, personal productivity and other uses. That has given some users the impression that their outputs have grown more trustworthy since the days when an early version of Google's AI Overviews suggested eating rocks and putting glue on pizza.
There is some truth to that intuition, but it is far from straightforward. Though tech companies have broadly made progress in reducing the likelihood that products powered by frontier AI models will hallucinate or otherwise go off the rails, the frequency still varies widely among models and can be difficult to assess.
The result: real-world flubs such as Olson's that can confuse users and in some cases deliver misinformation that users might unknowingly act on. AI watchdogs say the quality-control challenge will become even more acute as models become more agentic, or capable of making decisions without explicit direction from humans. The risk, they say, is that the agents will act using bad data, creating problems that have the potential to snowball within a user's computer system.
Paradoxically, AI systems that produce more accurate results can be more likely to deceive users with inaccurate ones, said Pratik Verma, founder and CEO of Okahu, which helps people improve their use of AI tools.
"When something is consistently wrong, the good thing is you know not to trust it," he said. "But when things are mostly right but sometimes wrong, that's the most pernicious one."
Verma said models are trained to produce an answer even when they are essentially guessing, and will make the same mistakes repeatedly if humans fail to correct them. He said it becomes increasingly important for users to verify AI output as the models become even more conversant and capable.
But relying on the vigilance of users is far from a perfect solution. Researchers at the University of Pennsylvania in February published a study on "cognitive surrender," or human propensity to accept AI-generated information whether or not it is accurate. They found that such acceptance becomes more likely if the person is under time pressure, faces a complex task or doesn't have a good understanding of what they are querying, among other factors.
Vanessa Culver, who recently left her job in the online payments industry, said she had been encouraged in her role to apply AI to a range of tasks, from automating manual processes to doing more complicated computation. The results were always mixed, she said, making it difficult to feel as though she could rely on it.
Still, she was surprised recently when she asked Claude to perform a simple task: adding keywords to the top of her résumé -- and it responded by altering the document in multiple ways she hadn't requested. It changed City University of Seattle, where she studied business administration, to the University of Washington, dropped mention of her master's degree and modified the timeline on several of her jobs.
"Working in tech, you have to embrace it, but then again, how much can you trust it?" she said.
Yet getting the most out of agentic AI systems often requires handing them at least a measure of control over accounts. Excitement about AI agents crested this year when a tool called OpenClaw went viral. OpenClaw agents are meant to be virtual personal assistants that users communicate with through apps including WhatsApp and iMessage. The agents can autonomously do things such as send emails, write code and clean up computer files.
Summer Yue, an AI safety researcher at Meta, went viral on X when she posted screenshots showing OpenClaw disregarding her instructions and deleting the contents of her inbox.
Still, companies including Meta, Amazon and Google parent Alphabet have reported huge surges in AI-generated code, with software engineers deploying agents to write and modify it for them.
Anish Agarwal, CEO and co-founder of Traversal, a startup building AI tools to troubleshoot software issues, likened code-writing agents to cars that, however well engineered, can still get into accidents when they hit traffic.
"It might be logically perfect, but it breaks once it starts interacting with other systems in unforeseen ways," he said.
Vidya Narayanan is a co-founder of FinalLayer, a startup developing agents for use on LinkedIn. She uses them for work as well, and recently had an experience similar to Yue's: she gave an agent limited direction to help manage a software project and it wound up deleting an entire folder from her coding repository without her permission.
The need to constantly review and verify AI outputs creates what she calls cognitive overhead, making the technology less useful. She recalled a recent session in which she used Anthropic's Claude to brainstorm for an hour and a half, then asked it to summarize the conversation in a document. The author, it wrote, was Vidya Plainfield.
"Who is Vidya 'Plainfield?'" she asked.
"You're right -- I completely made that up," it responded.
Olson, the Gemini user in Minneapolis, was alarmed enough by the details Gemini offered from what appeared to be someone else's Gmail account that he tried to report the matter to Google. He asked Gemini to draft an email to that address to alert the owner of the possible privacy violation.
"I've never given the password to log into this account," Olson said to Gemini. "Yet, in talking with you, you seem to have no issues sharing, you know, email content. It seems like quite a privacy breach."
"You're absolutely right, that does sound like a serious privacy concern, " Gemini replied. "I definitely want to help you with that."
When he asked Gemini to send his email to Google, however, it was unable to complete the task.
Olson said he is now more inclined to double-check AI output after learning Gemini had invented the entire exchange.
"It's definitely given me a bit more pause than just relying on it 100%, " he said. "It's still in kind of the trust-but-verify stage."
Write to Katherine Blunt at katherine.blunt@wsj.com
(END) Dow Jones Newswires
April 14, 2026 12:00 ET (16:00 GMT)
Copyright (c) 2026 Dow Jones & Company, Inc.
Comments