108 citations · 829 across the 38 of their papers we have counts for
41 papers
Towards Expert-level Medical AI for Real-time Video Consultations
Mahvish Nagda, Jihyeon Lee, Matthew Thompson +37
Audio-visual interaction is the standard for patient-physician consultations, enabling natural communication and effective assessment of illness through non-verbal cues. While text…
Towards Conversational Medical AI with Eyes, Ears and a Voice
Meet Shah, Jason Gusdorf, Anil Palepu +50
The practice of medicine relies not only upon skillful dialogue but also on the nuanced exchange and interpretation of rich auditory and visual cues between doctors and patients. B…
A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic
Peter Brodeur, Jacob M. Koshy, Anil Palepu +45
Large language model (LLM)-based AI systems have shown promise for patient-facing diagnostic and management conversations in simulated settings. Translating these systems into clin…
CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute
Chen Jin, Ryutaro Tanno, Tom Diethe +1
Large Language Models (LLMs) often rely on test-time scaling via parallel decoding (for example, 512 samples) to boost reasoning accuracy, but this incurs substantial compute. We i…
Complementary Human-AI Clinical Reasoning in Ophthalmology
Mertcan Sevgi, Fares Antaki, Abdullah Zafar Khan +26
Vision impairment and blindness are a major global health challenge where gaps in the ophthalmology workforce limit access to specialist care. We evaluate AMIE, a medically fine-tu…
Advancing Conversational Diagnostic AI with Multimodal Reasoning
Khaled Saab, Jan Freyberg, Chunjong Park +33
Large Language Models (LLMs) have demonstrated great potential for conducting diagnostic conversations but evaluation has been largely limited to language-only interactions, deviat…
Towards Conversational AI for Disease Management
Anil Palepu, Valentin Liévin, Wei-Hung Weng +17
While large language models (LLMs) have shown promise in diagnostic dialogue, their capabilities for effective management reasoning - including disease progression, therapeutic res…
Accelerating scientific discovery with Co-Scientist
Juraj Gottweis, Wei-Hung Weng, Alexander Daryin +48
Scientific discovery is driven by scientists generating novel hypotheses for complex problems that undergo rigorous experimental validation. To augment this process, we introduce C…
Diffusion Instruction Tuning
Chen Jin, Ryutaro Tanno, Amrutha Saseendran +2
We introduce Lavender, a simple supervised fine-tuning (SFT) method that boosts the performance of advanced vision-language models (VLMs) by leveraging state-of-the-art image gener…
Exploring Large Language Models for Specialist-level Oncology Care
Anil Palepu, Vikram Dhillon, Polly Niravath +18
Large language models (LLMs) have shown remarkable progress in encoding clinical knowledge and responding to complex medical queries with appropriate clinical reasoning. However, t…
Towards Democratization of Subspeciality Medical Expertise
Jack W. O'Sullivan, Anil Palepu, Khaled Saab +23
The scarcity of subspecialist medical expertise, particularly in rare, complex and life-threatening diseases, poses a significant challenge for healthcare delivery. This issue is p…
Advancing Multimodal Medical Capabilities of Gemini
Lin Yang, Shawn Xu, Andrew Sellergren +44
Many clinical tasks require an understanding of specialized data, such as medical images and genomics, which is not typically found in general-purpose large multimodal models. Buil…
Capabilities of Gemini Models in Medicine
Khaled Saab, Tao Tu, Wei-Hung Weng +64
Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of…
Towards Conversational Diagnostic AI
Tao Tu, Anil Palepu, Mike Schaekermann +22
At the heart of medicine lies the physician-patient dialogue, where skillful history-taking paves the way for accurate diagnosis, effective management, and enduring trust. Artifici…
Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
Ryutaro Tanno, David G. T. Barrett, Andrew Sellergren +23
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment. The worldwide shortage of radiologists, however, re…
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding
Talfan Evans, Shreya Pathak, Hamza Merzic +3
Power-law scaling indicates that large-scale training with uniform sampling is prohibitively slow. Active learning methods aim to increase data efficiency by prioritizing learning…
An Image is Worth Multiple Words: Discovering Object Level Concepts using Multi-Concept Prompt Learning
Chen Jin, Ryutaro Tanno, Amrutha Saseendran +2
Textural Inversion, a prompt learning method, learns a singular text embedding for a new "word" to represent image style and appearance, allowing it to be integrated into natural l…
Towards Generalist Biomedical AI
Tao Tu, Shekoofeh Azizi, Danny Driess +29
Medicine is inherently multimodal, with rich data modalities spanning text, imaging, genomics, and more. Generalist biomedical artificial intelligence (AI) systems that flexibly en…
Low-field magnetic resonance image enhancement via stochastic image quality transfer
Hongxiang Lin, Matteo Figini, Felice D'Arco +10
Low-field (<1T) magnetic resonance imaging (MRI) scanners remain in widespread use in low- and middle-income countries (LMICs) and are commonly used for some applications in higher…
Generative models improve fairness of medical classifiers under distribution shifts
Ira Ktena, Olivia Wiles, Isabela Albuquerque +9
A ubiquitous challenge in machine learning is the problem of domain generalisation. This can exacerbate bias against groups or labels that are underrepresented in the datasets used…
Repairing Neural Networks by Leaving the Right Past Behind
Ryutaro Tanno, Melanie F. Pradier, Aditya Nori +1
Prediction failures of machine learning models often arise from deficiencies in training data, such as incorrect labels, outliers, and selection biases. However, such data points t…
A Principled Approach to Failure Analysis and Model Repairment: Demonstration in Medical Imaging
Thomas Henn, Yasukazu Sakamoto, Clément Jacquet +8
Machine learning models commonly exhibit unexpected failures post-deployment due to either data shifts or uncommon situations in the training environment. Domain experts typically…
Learning to Downsample for Segmentation of Ultra-High Resolution Images
Chen Jin, Ryutaro Tanno, Thomy Mertzanidou +2
Many computer vision systems require low-cost segmentation algorithms based on deep learning, either because of the enormous size of input images or limited computational budget. C…
Active label cleaning for improved dataset quality under resource constraints
Melanie Bernhardt, Daniel C. Castro, Ryutaro Tanno +9
Imperfections in data annotation, known as label noise, are detrimental to the training of machine learning models and have an often-overlooked confounding effect on the assessment…