5 papers · 1 filter
Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs
Shayan Mohammadizadehsamakosh, Pritam Sarkar, Leonid Sigal +2
Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks, yet they remain prone to factual inconsistencies, poor visual grounding, and mis…
Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learning
Negin Baghbanzadeh, Mohammed Saidul Islam, Sajad Ashkezari +2
In biomedical vision-language modeling, datasets are typically mined from scientific literature, pairing compound figures with captions that are short, context-dependent, and ofter…
A Shared Encoder Approach to Multimodal Representation Learning
Shuvendu Roy, Franklin Ogidi, Ali Etemad +2
Multimodal representation learning has demonstrated remarkable potential in enabling models to process and integrate diverse data modalities, such as text and images, for improved…
Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning
Shuvendu Roy, Yasaman Parhizkar, Franklin Ogidi +5
We perform a comprehensive benchmarking of contrastive frameworks for learning multimodal representations in the medical domain. Through this study, we aim to answer the following…
Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Models
Shuvendu Roy, Elham Dolatabadi, Arash Afkanpour +1
We propose Consistency-guided Asynchronous Contrastive Tuning (CoACT), a novel method for continuously tuning foundation models to learn new classes in few-shot settings. CoACT con…