Publications (17)
Learning Sparse Visual Representations via Spatial-Semantic Factorization
Theodore Zhengde Zhao, Sid Kiblawi, Jianwei Yang +6
Self-supervised learning (SSL) faces a fundamental conflict between semantic understanding and image reconstruction. High-level semantic SSL (e.g., DINO) relies on global tokens th…
Scaling medical imaging report generation with multimodal reinforcement learning
Qianchu Liu, Sheng Zhang, Guanghui Qin +11
Frontier models have demonstrated remarkable capabilities in understanding and reasoning with natural-language text, but they still exhibit major competency gaps in multimodal unde…
Towards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision
Sam Preston, Mu Wei, Rajesh Rao +11
Objective: The majority of detailed patient information in real-world data (RWD) is only consistently available in free-text clinical documents. Manual curation is expensive and ti…
Universal Abstraction: Harnessing Frontier Models to Structure Real-World Data at Scale
Cliff Wong, Sam Preston, Qianchu Liu +22
A significant fraction of real-world patient information resides in unstructured clinical text. Medical abstraction extracts and normalizes key structured attributes from free-text…
X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains
Qianchu Liu, Sheng Zhang, Guanghui Qin +9
Recent proprietary models (e.g., o3) have begun to demonstrate strong multimodal reasoning capabilities. Yet, most existing open-source research concentrates on training text-only…
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs
Sheng Zhang, Yanbo Xu, Naoto Usuyama +21
Biomedical data is inherently multimodal, comprising physical measurements and natural language narratives. A generalist biomedical AI model needs to simultaneously process differe…
BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once
Theodore Zhao, Yu Gu, Jianwei Yang +12
Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. Holistic image analysis comprises interd…
AURAD: Anatomy-Pathology Unified Radiology Synthesis with Progressive Representations
Shuhan Ding, Jingjing Fu, Yu Gu +6
Medical image synthesis has become an essential strategy for augmenting datasets and improving model generalization in data-scarce clinical settings. However, fine-grained and cont…
Pareto Optimal Learning for Estimating Large Language Model Errors
Theodore Zhao, Mu Wei, J. Samuel Preston +1
Large Language Models (LLMs) have shown impressive abilities in many applications. When a concrete and precise answer is desired, it is important to have a quantitative estimation…
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu +24
The scaling laws and extraordinary performance of large foundation models motivate the development and utilization of such models in biomedicine. However, despite early promising r…
Boltzmann Attention Sampling for Image Analysis with Small Objects
Theodore Zhao, Sid Kiblawi, Naoto Usuyama +4
Detecting and segmenting small objects, such as lung nodules and tumor lesions, remains a critical challenge in image analysis. These objects often occupy less than 0.1% of an imag…
Distilling Large Language Models for Biomedical Knowledge Extraction: A Case Study on Adverse Drug Events
Yu Gu, Sheng Zhang, Naoto Usuyama +8
Large language models (LLMs), such as GPT-4, have demonstrated remarkable capabilities across a wide range of tasks, including health applications. In this paper, we study how LLMs…
MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging
Noel C. F. Codella, Ying Jin, Shrey Jain +28
In this work, we present MedImageInsight, an open-source medical imaging embedding model. MedImageInsight is trained on medical images with associated text and labels across a dive…
Foundation Models for Biomedical Image Segmentation: A Survey
Ho Hin Lee, Yu Gu, Theodore Zhao +9
Recent advancements in biomedical image analysis have been significantly driven by the Segment Anything Model (SAM). This transformative technology, originally developed for genera…
HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents
Qianchu Liu, Sheng Zhang, Guanghui Qin +16
As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare appli…
The Illusion of Readiness in Health AI
Yu Gu, Jingjing Fu, Xiaodong Liu +29
Large language models have demonstrated remarkable performance in a wide range of medical benchmarks. Yet underneath the seemingly promising results lie salient growth areas, espec…
CancerGUIDE: Cancer Guideline Understanding via Internal Disagreement Estimation
Alyssa Unell, Noel C. F. Codella, Sam Preston +13
The National Comprehensive Cancer Network (NCCN) provides evidence-based guidelines for cancer treatment. Translating complex patient presentations into guideline-compliant treatme…