6 papers
When Model Priors Conflict with Visual Evidence: Mitigating Commonsense-Driven Hallucinations by Selective Prior Calibration
Kesheng Chen, Yamin Hu, Wenjian Luo
In vision--language models, commonsense-driven hallucination (CDH) occurs when a model's commonsense prior overrides clear visual evidence of an atypical state. For example, a mode…
SentAttack: A Sentence-Level Black-Box Adversarial Attack Method for Dense Retrieval Models
Luping Wei, Yamin Hu, Sihan Shang +2
Retrieval-Augmented Generation (RAG) systems typically consist of a dense retrieval (DR) model for initial retrieval and a neural ranking model (NRM) for re-ranking.Existing robust…
AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging
Kesheng Chen, Yamin Hu, Zhenqian Zhu +2
LLM services need to offer a family of models spanning different capability--cost trade-offs to accommodate diverse user preferences. Model merging offers a practical way to constr…
Rethinking Backdoor Adversarial Unlearning through the Lens of Catastrophic Forgetting in Continual Learning
Zhenqian Zhu, Yamin Hu, Yujiang Liu +5
Existing studies reveal that current backdoor defenses exhibit limited robustness and often fail against specific types of attacks. More concerningly, prevailing safety tuning stra…
From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging
Zhenqian Zhu, Yamin Hu, Yiya Diao +3
Model merging (MM) has gained significant attention as a cost-effective approach to integrate multiple task-specific models into a unified model. However, recent work reveals that…
CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
Kesheng Chen, Yamin Hu, Qi Zhou +2
Vision-language models (VLMs) achieve strong performance on many benchmarks, yet a basic reliability question remains underexplored: when visual evidence conflicts with commonsense…