12 papers
Grounding Isn't Knowing: Do VLMs Need Object Localization for Spatial Reasoning?
Xiwei Liu, Yulong Li, Xinlin Zhuang +5
Vision-language models (VLMs) can answer spatial questions, yet the mechanisms connecting object grounding to spatial reasoning remain poorly understood. It is underexplored whethe…
Counterfactual Anatomy-guided Spatial-Temporal Decoding for Annotation-Free Hallucination Mitigation in Medical VLMs
Yifan Lu, Adinath Dukre, Abhijit Das +4
Medical vision-language models (Med-VLMs) have demonstrated strong performance on medical visual question answering, yet they remain prone to hallucination, generating clinically u…
Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers
Hao Yan, Haolin Yang, Yiqiao Zhong
Transformers are effective at inferring the latent task from context via two inference modes: recognizing a task seen during training, and adapting to a novel one. Recent interpret…
Project Imaging-X: A Survey of 1000+ Open-Access Medical Imaging Datasets for Foundation Model Development
Zhongying Deng, Cheng Tang, Ziyan Huang +124
Foundation models have demonstrated remarkable success across diverse domains and tasks, primarily due to the thrive of large-scale, diverse, and high-quality datasets. However, in…
ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models
Xiwei Liu, Yulong Li, Xinlin Zhuang +5
Medical Vision-Language Models have shown promising potential in clinical decision support, yet they remain prone to factual hallucinations due to insufficient grounding in localiz…
APEX: Learning Adaptive Priorities for Multi-Objective Alignment in Vision-Language Generation
Dongliang Chen, Xinlin Zhuang, Junjie Xu +8
Multi-objective alignment for text-to-image generation is commonly implemented via static linear scalarization, but fixed weights often fail under heterogeneous rewards, leading to…
CHIPS: Efficient CLIP Adaptation via Curvature-aware Hybrid Influence-based Data Selection
Xinlin Zhuang, Yichen Li, Xiwei Liu +11
Adapting CLIP to vertical domains is typically approached by novel fine-tuning strategies or by continual pre-training (CPT) on large domain-specific datasets. Yet, data itself rem…
Towards Robust Visual Continual Learning with Multi-Prototype Supervision
Xiwei Liu, Yulong Li, Yichen Li +4
Language-guided supervision, which utilizes a frozen semantic target from a Pretrained Language Model (PLM), has emerged as a promising paradigm for visual Continual Learning (CL).…
scAGC: Learning Adaptive Cell Graphs with Contrastive Guidance for Single-Cell Clustering
Huifa Li, Jie Fu, Xinlin Zhuang +6
Accurate cell type annotation is a crucial step in analyzing single-cell RNA sequencing (scRNA-seq) data, which provides valuable insights into cellular heterogeneity. However, due…
StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding
Haolin Yang, Feilong Tang, Lingxiao Zhao +10
Real-time streaming video understanding in domains such as autonomous driving and intelligent surveillance poses challenges beyond conventional offline video processing, requiring…
Towards Efficient Medical Reasoning with Minimal Fine-Tuning Data
Xinlin Zhuang, Feilong Tang, Haolin Yang +9
Supervised Fine-Tuning (SFT) of the language backbone plays a pivotal role in adapting Vision-Language Models (VLMs) to specialized domains such as medical reasoning. However, exis…
ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videos
Haolin Yang, Feilong Tang, Ming Hu +8
Video diffusion models (VDMs) facilitate the generation of high-quality videos, with current research predominantly concentrated on scaling efforts during training through improvem…