9 papers
When Can Test-Time Adaptation Help Zero-Shot CT Vision-Language Models?
Ailar Mahdizadeh, Puria Azadi Moghadam, Xiangteng He +1
3D CT vision-language models (VLMs) classify abnormalities from text prompts in a zero-shot manner, enabling cross-institution deployment where labels are scarce and clinical tasks…
Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs
Shayan Mohammadizadehsamakosh, Pritam Sarkar, Leonid Sigal +2
Preference optimization is increasingly used to post-train medical large vision-language models (LVLMs), yet it operates at a much coarser granularity than the one that defines cli…
SCALE-VLP: Soft-Weighted Contrastive Volumetric Vision-Language Pre-training with Spatial-Knowledge Semantics
Ailar Mahdizadeh, Puria Azadi Moghadam, Xiangteng He +3
Vision-language models (VLMs) have demonstrated strong cross-modal capabilities, yet most work remains limited to 2D data and assumes binary supervision (i.e., positive vs. negativ…
DSeq-JEPA: Discriminative Sequential Joint-Embedding Predictive Architecture
Xiangteng He, Shunsuke Sakai, Shivam Chandhok +5
Recent advances in self-supervised visual representation learning have demonstrated the effectiveness of predictive latent-space objectives for learning transferable features. In p…
To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
Jiayun Luo, Wan-Cyuan Fan, Lyuyang Wang +4
Large Vision Language Models (LVLMs) have recently emerged as powerful architectures capable of understanding and reasoning over both visual and textual information. These models t…
InvAD: Inversion-based Reconstruction-Free Anomaly Detection with Diffusion Models
Shunsuke Sakai, Xiangteng He, Chunzhi Gu +2
Despite the remarkable success, recent reconstruction-based anomaly detection (AD) methods via diffusion modeling still involve fine-grained noise-strength tuning and computational…
MMFactory: A Universal Solution Search Engine for Vision-Language Tasks
Wan-Cyuan Fan, Tanzila Rahman, Leonid Sigal
With advances in foundational and vision-language models, and effective fine-tuning techniques, a large number of both general and special-purpose models have been developed for a…
Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
Tanzila Rahman, Shweta Mahajan, Hsin-Ying Lee +3
Text-to-image (TTI) diffusion models have demonstrated impressive results in generating high-resolution images of complex and imaginative scenes. Recent approaches have further ext…
Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models
Shweta Mahajan, Tanzila Rahman, Kwang Moo Yi +1
The quality of the prompts provided to text-to-image diffusion models determines how faithful the generated content is to the user's intent, often requiring `prompt engineering'. T…