6 papers
Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs
Shayan Mohammadizadehsamakosh, Pritam Sarkar, Leonid Sigal +2
Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks, yet they remain prone to factual inconsistencies, poor visual grounding, and mis…
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We i…
LookWhen? Fast Video Recognition by Learning When, Where, and What to Compute
Ali Salamatian, Anthony Fuller, Pritam Sarkar +3
Transformers dominate video recognition. They split videos into tokens, and processing them has expensive superlinear computational cost. Yet videos are filled with redundancy, so…
Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization
Pritam Sarkar, Ali Etemad
Despite recent advances in Large Video Language Models (LVLMs), they still struggle with fine-grained temporal understanding, hallucinate, and often make simple mistakes on even si…
VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models
Pritam Sarkar, Ali Etemad
Despite recent advances in video understanding, the capabilities of Large Video Language Models (LVLMs) to perform video-based causal reasoning remains underexplored, largely due t…
Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment
Pritam Sarkar, Sayna Ebrahimi, Ali Etemad +3
Despite their significant advancements, Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucination. In this work, we add…