6 papers
Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination
Yangneng Chen, Junlin Li, Weijun Yao +4
Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal tasks, yet their reliability is persistently undermined by hallucinations-generating text that…
Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework
Xilai Ma, Liye Zhao, Weijun Yao +3
Large Language Model (LLM) personalization aims to align model behaviors with individual user preferences. Existing methods often focus on isolated user histories, neglecting the e…
Team-Based Self-Play With Dual Adaptive Weighting for Fine-Tuning LLMs
Wu Li, Yigeng Zhou, Zesheng Shi +3
While recent self-training approaches have reduced reliance on human-labeled data for aligning LLMs, they still face critical limitations: (i) sensitivity to synthetic data quality…
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
Jiahui Guang, Zexun Zhan, Zhenlin Xu +5
Multimodal Large Language Models (MLLMs) may memorize sensitive cross-modal information during pretraining. However, existing MLLM unlearning benchmarks rely on synthetic knowledge…
Unveiling Modality Bias: Automated Sample-Specific Analysis for Multimodal Misinformation Benchmarks
Hehai Lin, Hui Liu, Shilei Cao +3
Numerous multimodal misinformation benchmarks exhibit bias toward specific modalities, allowing detectors to make predictions based solely on one modality. While previous research…
STARE at the Structure: Steering ICL Exemplar Selection with Structural Alignment
Jiaqian Li, Qisheng Hu, Jing Li +1
In-Context Learning (ICL) has become a powerful paradigm that enables LLMs to perform a wide range of tasks without task-specific fine-tuning. However, the effectiveness of ICL hea…