Google is testing an artificial-intelligence program trained to expertly answer medical questions, racing against rivals including Microsoft to translate recent AI advances into products that would be used widely by clinicians.
The November release of ChatGPT, a computer program that can fluently respond to a range of queries across subjects, has sparked early experiments at health systems across the U.S. to use the underlying technology in patient care.
Google is betting that its medical chatbot technology, which is called Med-PaLM 2, will be better at holding conversations on healthcare issues than more general-purpose algorithms because it has been fed questions and answers from medical licensing exams. The company began testing the system with customers including the research hospital Mayo Clinic in April, said people familiar with the matter.
Med-PaLM 2 can be used to generate responses to medical questions and perform tasks such as summarizing documents or organizing reams of health data, according to Google executives and research published by the company.
The healthcare industry has become a new front in the battle between big tech companies and smaller startups to win customers with AI offerings, though past efforts such as IBM's Watson Health initiative have sometimes struggled to translate the technology into lasting profits.
Medical leaders and ethicists said that while generative AI could be transformative for medicine, patients must be told about any new ways their health data is being used, and new tools must be evaluated as they are rolled out. Google , a unit of Alphabet, has drawn scrutiny in the past for how it handles sensitive health data through its partnerships with hospitals.
AI algorithms are already used in hospitals for specialized tasks, such as predicting heart trouble from patient electrocardiograms. Generative AI tools present a new set of risks because they can be used to produce authoritative-sounding responses to medical questions, potentially influencing patients in ways that doctors wouldn't endorse.
Google executives said customers testing Med-PaLM 2 would retain control of their data in encrypted settings inaccessible to the tech company, and the program wouldn't ingest any of that data.
A Google spokeswoman declined to say when the program would be made more widely available to customers or the general public.
Google's rivals have moved quickly to incorporate AI advances into patient interactions. Microsoft, the largest investor in OpenAI and its closest business partner, in April teamed up with the health software company Epic to build tools that can automatically draft messages to patients using the algorithms behind ChatGPT.
Those offerings could boost the companies' cloud-computing businesses, an area of focus for the tech giants as they promote the potential of AI programs. Google opened an office in 2021 in Rochester, Minn. -- near the Mayo Clinic's headquarters -- to work on projects using the hospital's data. The hospital said in June that it would use Google AI models to build a new internal search tool for querying patient records.
Both Google and Microsoft also have expressed interest in a bigger ambition: building a virtual assistant that answers medical questions from patients around the world, particularly in areas with limited resources, according to company documents.
Google told employees in April that an AI model trusted as a medical assistant could "be of tremendous value in countries that have more limited access to doctors," according to an internal email reviewed by The Wall Street Journal that quotes a researcher working on the project.
Microsoft and OpenAI said in a paper released in March that algorithms such as the GPT-4 program behind ChatGPT "could be harnessed to provide information, communication, screening, and decision support in under-served regions."
Greg Corrado, a senior research director at Google who worked on Med-PaLM 2, said the company was still in the early stages of developing products using the technology and working with customers to understand their needs.
"I don't feel that this kind of technology is yet at a place where I would want it in my family's healthcare journey," Corrado said. However, Med-PaLM 2 "takes the places in healthcare where AI can be beneficial and expands them by 10-fold," he said.
Google has often held back some of its most advanced AI programs from the general public because of concerns about their safety and potential impact on the core online search business. That caution provided an opening for Microsoft and OpenAI, which have moved more quickly to release the popular ChatGPT chatbot to the public and provide customers access to the underlying AI systems.
Hospitals are beginning to test OpenAI's GPT algorithms through Microsoft's cloud service, in tasks such as summarizing doctors' notes or generating reminders. Microsoft hosts and controls the AI systems in these cases, a spokeswoman said. Google's Med-PaLM 2 and OpenAI's GPT-4 each scored similarly on medical exam questions, according to independent research released by the companies.
Doctors and healthcare executives said programs such as Med-PaLM 2 still needed more development and testing before being used to diagnose patients and suggest treatments.
In May, the World Health Organization said it was concerned "caution that would normally be exercised for any new technology is not being exercised consistently with LLMs," referring to the large language models powering chatbots.
Physicians who reviewed answers provided by Med-PaLM 2 to more than 1,000 consumer medical questions preferred the system's responses to those produced by doctors along eight out of nine categories for evaluation defined by Google, according to research the company made public in May.
However, the doctors found Med-PaLM 2 included more inaccurate or irrelevant content in its responses than those of their peers, suggesting the program shares similar issues with other chatbots that have a tendency to confidently generate off-topic or false statements.
Google researchers said there wasn't a significant improvement in the program's ability to avoid inaccurate or irrelevant information from the first version announced in December.
"We don't have a way of evaluating these things at scale," said Dev Dash, a clinical assistant professor at Stanford University School of Medicine who has researched applications of AI in medicine. "It's very much a work in progress."
Kellie Owens, a medical ethicist at NYU Grossman School of Medicine, said patients should be educated on any new ways their health data is used by AI tools.
"These need to be conversations human-to-human," ideally between patients and doctors, or medical staff, rather than a disclosure buried in a consent form, Owens said.
Both Google and Microsoft said they don't use patient data to train their algorithms. Corrado said Google might allow healthcare companies to create custom versions of Med-PaLM 2 using patient records and other data, but that isn't currently possible.
