8 papers · 1 filter
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We i…
Tinted Frames: Question Framing Blinds Vision-Language Models
Wan-Cyuan Fan, Jiayun Luo, Declan Kutscher +2
Vision-Language Models (VLMs) have been shown to be blind, often underutilizing their visual inputs even on tasks that require visual reasoning. In this work, we demonstrate that V…
Learning What Matters: Prioritized Concept Learning via Relative Error-driven Sample Selection
Shivam Chandhok, Qian Yang, Oscar Manas +3
Instruction tuning has been central to the success of recent vision-language models (VLMs), but it remains expensive-requiring large-scale datasets, high-quality annotations, and l…
On Pre-training of Multimodal Language Models Customized for Chart Understanding
Wan-Cyuan Fan, Yen-Chun Chen, Mengchen Liu +2
Recent studies customizing Multimodal Large Language Models (MLLMs) for domain-specific tasks have yielded promising results, especially in the field of scientific chart comprehens…
Test-Time Consistency in Vision Language Models
Shih-Han Chou, Shivam Chandhok, James J. Little +1
Vision-Language Models (VLMs) have achieved impressive performance across a wide range of multimodal tasks, yet they often exhibit inconsistent behavior when faced with semanticall…
MM-R: On (In-)Consistency of Vision-Language Models (VLMs)
Shih-Han Chou, Shivam Chandhok, James J. Little +1
With the advent of LLMs and variants, a flurry of research has emerged, analyzing the performance of such models across an array of tasks. While most studies focus on evaluating th…