7 papers
Visual Attention Drifts,but Anchors Hold:Mitigating Hallucination in Multimodal Large Language Models via Cross-Layer Visual Anchors
Chengxu Yang, Jingling Yuan, Chuang Hu +1
Multimodal Large Language Models often suffer from object hallucination. While existing research utilizes attention enhancement and visual retracing, we find these works lack suffi…
SAGE: Accelerating Vision-Language Models via Entropy-Guided Adaptive Speculative Decoding
Yujia Tong, Tian Zhang, Yunyang Wan +3
Speculative decoding has emerged as a promising approach to accelerate inference in vision-language models (VLMs) by enabling parallel verification of multiple draft tokens. Howeve…
Heaven-Sent or Hell-Bent? Benchmarking the Intelligence and Defectiveness of LLM Hallucinations
Chengxu Yang, Jingling Yuan, Siqi Cai +2
Hallucinations in large language models (LLMs) are commonly regarded as errors to be minimized. However, recent perspectives suggest that some hallucinations may encode creative or…
LetheViT: Selective Machine Unlearning for Vision Transformers via Attention-Guided Contrastive Learning
Yujia Tong, Tian Zhang, Jingling Yuan +2
Vision Transformers (ViTs) have revolutionized computer vision tasks with their exceptional performance. However, the introduction of privacy regulations such as GDPR and CCPA has…
DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning
Yujia Tong, Jingling Yuan, Tian Zhang +2
Data-Free Quantization (DFQ) enables the quantization of Vision Transformers (ViTs) without requiring access to data, allowing for the deployment of ViTs on devices with limited re…
Enhancing Quantization-Aware Training on Edge Devices via Relative Entropy Coreset Selection and Cascaded Layer Correction
Yujia Tong, Jingling Yuan, Chuang Hu
With the development of mobile and edge computing, the demand for low-bit quantized models on edge devices is increasing to achieve efficient deployment. To enhance the performance…