6 papers
HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression
Yuxuan Yang, Feiyang Ren, Bowen Zeng +4
Long-context LLM inference faces a fundamental conflict: head-adaptive compression algorithms (e.g., Top- nucleus sampling) offer superior accuracy by dynamically fluctuating me…
Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs
Xianya Fang, Feiyang Ren, Xiang Chen +4
Multimodal LLMs are powerful but prone to object hallucinations, which describe non-existent entities and harm reliability. While recent unlearning methods attempt to mitigate this…
Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
Jun Zhang, Yicheng Ji, Feiyang Ren +7
Large Vision-Language Models (LVLMs) enable sophisticated reasoning over images and videos, yet their inference is hindered by a systemic efficiency barrier known as visual token d…
HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
Bowen Zeng, Feiyang Ren, Jun Zhang +4
Multimodal Large Language Models (MLLMs) have advanced unified reasoning over text, images, and videos, but their inference is hindered by the rapid growth of key-value (KV) caches…
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
Zijian Zhu, Fei Ren, Zhanhong Tan +1
Diffusion large language models (dLLMs) are emerging as a promising alternative to autoregressive models (ARMs) due to their ability to capture bidirectional context and the potent…
SpeCa: Accelerating Diffusion Transformers with Speculative Feature Caching
Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu +4
Diffusion models have revolutionized high-fidelity image and video synthesis, yet their computational demands remain prohibitive for real-time applications. These models face two f…