1 citations · 3 across the 9 of their papers we have counts for
12 papers
SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
Ahmadreza Jeddi, Enming Zhang, Jasper Gerigk +12
Reasoning vision-language models (VLMs) process images and videos as long sequences of visual tokens, making inference expensive. Training-free token pruning reduces this cost, but…
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We i…
Tinted Frames: Question Framing Blinds Vision-Language Models
Wan-Cyuan Fan, Jiayun Luo, Declan Kutscher +2
Vision-Language Models (VLMs) have been shown to be blind, often underutilizing their visual inputs even on tasks that require visual reasoning. In this work, we demonstrate that V…
In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding
Wan-Cyuan Fan, Yen-Chun Chen, Mengchen Liu +3
Recent methods for customizing Large Vision Language Models (LVLMs) for domain-specific tasks have shown promising results in scientific chart comprehension. However, existing appr…
Response Wide Shut? Surprising Observations in Basic Vision Language Model Capabilities
Shivam Chandhok, Wan-Cyuan Fan, Vered Shwartz +2
Vision-language Models (VLMs) have emerged as general-purpose tools for addressing a variety of complex computer vision problems. Such models have been shown to be highly capable,…
Test-Time Consistency in Vision Language Models
Shih-Han Chou, Shivam Chandhok, James J. Little +1
Vision-Language Models (VLMs) have achieved impressive performance across a wide range of multimodal tasks, yet they often exhibit inconsistent behavior when faced with semanticall…
Learning What Matters: Prioritized Concept Learning via Relative Error-driven Sample Selection
Shivam Chandhok, Qian Yang, Oscar Manas +3
Instruction tuning has been central to the success of recent vision-language models (VLMs), but it remains expensive-requiring large-scale datasets, high-quality annotations, and l…
ADiff4TPP: Asynchronous Diffusion Models for Temporal Point Processes
Amartya Mukherjee, Ruizhi Deng, He Zhao +3
This work introduces a novel approach to modeling temporal point processes using diffusion models with an asynchronous noise schedule. At each step of the diffusion process, the no…
MM-R: On (In-)Consistency of Vision-Language Models (VLMs)
Shih-Han Chou, Shivam Chandhok, James J. Little +1
With the advent of LLMs and variants, a flurry of research has emerged, analyzing the performance of such models across an array of tasks. While most studies focus on evaluating th…
Response Wide Shut: Surprising Observations in Basic Vision Language Model Capabilities
Shivam Chandhok, Wan-Cyuan Fan, Leonid Sigal
Vision-Language Models (VLMs) have emerged as general purpose tools for addressing a variety of complex computer vision problems. Such models have been shown to be highly capable,…
On Pre-training of Multimodal Language Models Customized for Chart Understanding
Wan-Cyuan Fan, Yen-Chun Chen, Mengchen Liu +2
Recent studies customizing Multimodal Large Language Models (MLLMs) for domain-specific tasks have yielded promising results, especially in the field of scientific chart comprehens…
Multi-modal News Understanding with Professionally Labelled Videos (ReutersViLNews)
Shih-Han Chou, Matthew Kowal, Yasmin Niknam +8
While progress has been made in the domain of video-language understanding, current state-of-the-art algorithms are still limited in their ability to understand videos at high leve…