Just now, a new breakthrough in AI healthcare has emerged from Google (GOOGL.US), and this time, they are directly targeting the pain points of real clinical environments. For a long time, medical models have been like "students who are good at only one subject"; they excel at "reading medical records" but struggle with medical images like CT scans, MRIs, and pathology slides. This is because they are forced to use text-based logic to understand images, leading to low efficiency, numerous errors, and high costs. To address this, Google has unveiled its latest model, MedGemma 1.5, which provides a solution. Compared to its predecessor, MedGemma 1.5 achieves a major breakthrough in multimodal applications, integrating: high-dimensional medical imaging (Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and histopathology); longitudinal medical imaging (reviewing chest X-ray time series); anatomical localization (locating anatomical features in chest X-rays); and medical document understanding (extracting structured data from medical laboratory reports). Google states that MedGemma 1.5 is the first publicly released open-source multimodal large language model capable of interpreting high-dimensional medical data while also retaining the ability to interpret general 2D images and text. More crucially, MedGemma 1.5 has only 4 billion parameters, meaning it can run smoothly on standard consumer-grade graphics cards or even high-performance workstations. Furthermore, Google also released MedASR, a speech recognition model specifically fine-tuned for medical speech, which can convert doctor-patient conversations into text and seamlessly integrate with MedGemma. In simple terms, MedGemma 1.5 solves "how to see images," while MedASR solves "how to hear sounds." This is not merely a simple model iteration but rather a systematic answer from Google to the question of "how to truly bring AI into the examination room." An AI doctor that can thoroughly read medical records, accurately understand images, and clearly interpret speech is poised to enter every hospital. AI healthcare is entering the multimodal era. Over the past year, we have witnessed impressive performances by models like GPT-5 on medical exams, but their performance in real clinical settings has often been disappointing. A key reason is the disconnect in information dimensions. Many medical models, including the first-generation MedGemma, are essentially "text experts" with weak image comprehension capabilities, leading to a loss of diagnostic information. MedGemma 1.5, however, achieves a comprehensive, multi-dimensional performance leap in medical imaging applications, significantly surpassing its predecessor. For high-dimensional medical imaging, MedGemma 1.5 achieves: an increase in CT disease classification accuracy from 58% to 61%; an increase in MRI disease classification accuracy from 51% to 65%, with notable progress particularly in recognizing complex anatomical structures like the brain and joints; and the quality score for whole-slide pathology descriptions (ROUGE-L) improves from a nearly ineffective 0.02 to 0.49, reaching the level of the specialized model PolyPath (0.498), enabling the generation of clinically usable histological descriptions. For longitudinal time-series image analysis, MedGemma 1.5 achieves: an increase in macro accuracy on the MS-CXR-T temporal evaluation benchmark from 61% to 66%; it effectively captures dynamic changes in lesions, such as judging whether pneumonia infiltration has resolved, supporting follow-up decision-making. For general 2D medical image interpretation, MedGemma 1.5 achieves: an overall classification accuracy increase from 59% to 62% on an internal comprehensive single-image benchmark (covering X-rays, skin, fundus, and pathology slides), indicating the model maintains broad 2D capabilities without sacrificing basic performance due to the addition of high-dimensional tasks. For structured medical documents, MedGemma 1.5 achieves: an increase in the macro-average F1 score for extracting test items, values, and units from unstructured PDFs or text from 60% to 78% (+18 percentage points), automatically building structured databases and finalizing the integration of multi-source information analysis combining images, text, and lab tests. Simultaneously, traditional Automatic Speech Recognition (ASR) models, when faced with rare medical terminology, perform like complete medical novices, with extremely high word error rates turning AI data entry into a burden for doctors. The newly released MedASR model, fine-tuned specifically for healthcare, shows a significant reduction in error rates. Researchers compared MedASR's performance with the general-purpose ASR model Whisper large-v3. They found that MedASR reduced error rates in chest X-ray dictations by 58% and reduced errors in dictations across different specialties by 82%. Trillion-dollar Google is betting heavily on AI healthcare. Google's strategic footprint in the healthcare sector is extensive, with its technological reach extending into every corner of the industry. Regarding investments, Google, through its venture capital and private equity arms, has invested in numerous life sciences companies. AI drug discovery has become a key focus area favored by Google; out of 51 healthcare investments by Google Ventures in 2021, 28 were in drug R&D, accounting for more than half. On the collaboration front, leveraging its industry-leading services in artificial intelligence, cloud computing, and more, Google has in recent years struck deals with pharmaceutical companies and hospitals like Bayer, Pfizer, Servier, and the Mayo Clinic to explore intelligent solutions spanning from drug discovery to clinical diagnosis and treatment. Internally, besides Google Health, Google has business units like Verily and Calico focused on different domains, forming a diverse and powerful matrix. Notably, as a world-leading AI research institution, Google DeepMind has launched several scientifically significant models, including AlphaFold (protein structure), AlphaGenome (DNA regulation), and C2S-Scale (single-cell). DeepMind's CEO, Demis Hassabis, was awarded the 2024 Nobel Prize in Chemistry for his contributions to AI-based protein structure prediction. In recent years, riding the wave of large language models, Google has also developed several vertical-specific large models for healthcare. These models can not only help doctors diagnose diseases more accurately but also provide patients with personalized health advice. The Google team first developed Flan-PaLM, a model that tackled the US Medical Licensing Examination (USMLE) and achieved a score of 67.6%, a 17-percentage-point improvement over the previous best model. Subsequently, Google released Med-PaLM, a result published in the journal *Nature*, which, when judged by professional clinicians, demonstrated answer accuracy comparable to that of humans for practical questions. In 2023, the world's first generalist medical large model, Med-PaLM M, was released, performing close to or exceeding the existing state-of-the-art (SOTA) across 14 benchmark tasks (including question answering, report generation and summarization, visual question answering, medical image classification, and genomic variant calling). Last year, Google's Chief Health Officer, Dr. Karen DeSalvo, announced six advancements, including the AI drug discovery model TxGemma, an FDA-cleared watch-based pulse stoppage detection feature, the multi-agent system "AI Collaboratory Scientist," and a pediatric personalized cancer treatment model. From medical imaging to drug discovery, and from health assistants to wearable devices, Google is redefining the future of medicine.
Comments