2.2k citations · 6.2k across the 59 of their papers we have counts for
61 papers
Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models
Yuxin Xiao, Sheng Zhang, Chandan Singh +4
Electronic health record (EHR) foundation models trained on longitudinal patient trajectories have demonstrated strong performance across diverse clinical prediction tasks. However…
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
Qin Liu, Wenxuan Zhou, Nan Xu +5
One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively sel…
Semantic-Clipping: Efficient Vision-Language Modeling with Semantic-Guidedd Visual Selection
Bangzheng Li, Fei Wang, Wenxuan Zhou +5
Vision-Language Models (VLMs) leverage aligned visual encoders to transform images into visual tokens, allowing them to be processed similarly to text by the backbone large languag…
Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learning
Sheng Zhang, Qianchu Liu, Guanghui Qin +2
Reinforcement learning from verifiable rewards (RLVR) has recently gained attention for its ability to elicit self-evolved reasoning capabilitie from base language models without e…
MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging
Noel C. F. Codella, Ying Jin, Shrey Jain +28
In this work, we present MedImageInsight, an open-source medical imaging embedding model. MedImageInsight is trained on medical images with associated text and labels across a dive…
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
Fei Wang, Xingyu Fu, James Y. Huang +18
We introduce MuirBench, a comprehensive benchmark that focuses on robust multi-image understanding capabilities of multimodal LLMs. MuirBench consists of 12 diverse multi-image tas…
From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning
Nan Xu, Fei Wang, Sheng Zhang +2
Motivated by in-context learning (ICL) capabilities of Large Language Models (LLMs), multimodal LLMs with additional visual modality are also exhibited with similar ICL abilities w…
BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once
Theodore Zhao, Yu Gu, Jianwei Yang +12
Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. Holistic image analysis comprises interd…
mDPO: Conditional Preference Optimization for Multimodal Large Language Models
Fei Wang, Wenxuan Zhou, James Y. Huang +4
Direct preference optimization (DPO) has shown to be an effective method for large language model (LLM) alignment. Recent works have attempted to apply DPO to multimodal scenarios…
Offset Unlearning for Large Language Models
James Y. Huang, Wenxuan Zhou, Fei Wang +4
Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as c…
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu +24
The scaling laws and extraordinary performance of large foundation models motivate the development and utilization of such models in biomedicine. However, despite early promising r…
Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summaries
Zelalem Gero, Chandan Singh, Yiqing Xie +6
Summarizing clinical text is crucial in health decision-support and clinical research. Large language models (LLMs) have shown the potential to generate accurate clinical text summ…
T-Rex: Text-assisted Retrosynthesis Prediction
Yifeng Liu, Hanwen Xu, Tangqi Fang +5
As a fundamental task in computational chemistry, retrosynthesis prediction aims to identify a set of reactants to synthesize a target molecule. Existing template-free approaches o…
Foundation Models for Biomedical Image Segmentation: A Survey
Ho Hin Lee, Yu Gu, Theodore Zhao +9
Recent advancements in biomedical image analysis have been significantly driven by the Segment Anything Model (SAM). This transformative technology, originally developed for genera…
When an Image is Worth 1,024 x 1,024 Words: A Case Study in Computational Pathology
Wenhui Wang, Shuming Ma, Hanwen Xu +4
This technical report presents LongViT, a vision Transformer that can process gigapixel images in an end-to-end manner. Specifically, we split the gigapixel image into a sequence o…
Cautionary Tales on Synthetic Controls in Survival Analyses
Alicia Curth, Hoifung Poon, Aditya V. Nori +1
Synthetic control (SC) methods have gained rapid popularity in economics recently, where they have been applied in the context of inferring the effects of treatments on standard co…
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
Harsha Nori, Yin Tat Lee, Sheng Zhang +15
Generalist foundation models such as GPT-4 have displayed surprising capabilities in a wide variety of domains and tasks. Yet, there is a prevalent assumption that they cannot matc…
DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generation
Yiqing Xie, Sheng Zhang, Hao Cheng +6
Medical text generation aims to assist with administrative work and highlight salient information to support decision-making. To reflect the specific requirements of medical text,…
TRIALSCOPE: A Unifying Causal Framework for Scaling Real-World Evidence Generation with Biomedical Language Models
Javier González, Risa Ueno, Cliff Wong +12
The rapid digitization of real-world data presents an unprecedented opportunity to optimize healthcare delivery and accelerate biomedical discovery. However, these data are often f…
Exploring the Boundaries of GPT-4 in Radiology
Qianchu Liu, Stephanie Hyland, Shruthi Bannur +16
The recent success of general-domain large language models (LLMs) has significantly changed the natural language processing paradigm towards a unified foundation model across domai…
BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys
Yu Gu, Jianwei Yang, Naoto Usuyama +5
Rapid progress has been made in instruction-learning for image editing with natural-language instruction, as exemplified by InstructPix2Pix. In biomedicine, such methods can be app…
Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology
Cliff Wong, Sheng Zhang, Yu Gu +8
Clinical trial matching is a key process in health delivery and discovery. In practice, it is plagued by overwhelming unstructured data and unscalable manual processing. In this pa…
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition
Wenxuan Zhou, Sheng Zhang, Yu Gu +2
Large language models (LLMs) have demonstrated remarkable generalizability, such as understanding arbitrary entities and relations. Instruction tuning has proven effective for dist…
Distilling Large Language Models for Biomedical Knowledge Extraction: A Case Study on Adverse Drug Events
Yu Gu, Sheng Zhang, Naoto Usuyama +8
Large language models (LLMs), such as GPT-4, have demonstrated remarkable capabilities across a wide range of tasks, including health applications. In this paper, we study how LLMs…