5 papers
Mixture-of-Memories: Low-Overhead Language Modeling with Modular Knowledge Injection
Yifan Lu, Rubin Wei, Ziyun Zou +8
Large language models store substantial factual knowledge in their parameters, yet this knowledge is difficult to control, update, and reliably elicit at inference time. Existing m…
Think Before You Locate: Clinical Chain-of-Though Supervision for Sequential Medical Image Grounding
Adinath Madhavrao Dukre, Govinda Kolli, Yifan Lu +7
Clinicians rarely localize a finding from one image alone. They compare current and prior scans, follow structures across slices, and match regions across views or modalities. Yet…
Counterfactual Anatomy-guided Spatial-Temporal Decoding for Annotation-Free Hallucination Mitigation in Medical VLMs
Yifan Lu, Adinath Dukre, Abhijit Das +4
Medical vision-language models (Med-VLMs) have demonstrated strong performance on medical visual question answering, yet they remain prone to hallucination, generating clinically u…
CHIPS: Efficient CLIP Adaptation via Curvature-aware Hybrid Influence-based Data Selection
Xinlin Zhuang, Yichen Li, Xiwei Liu +11
Adapting CLIP to vertical domains is typically approached by novel fine-tuning strategies or by continual pre-training (CPT) on large domain-specific datasets. Yet, data itself rem…
HeSRN: Representation Learning On Heterogeneous Graphs via Slot-Aware Retentive Network
Yifan Lu, Ziyun Zou, Belal Alsinglawi +6
Graph Transformers have recently achieved remarkable progress in graph representation learning by capturing long-range dependencies through self-attention. However, their quadratic…