4 papers
Beyond Token Eviction: Mixed-Dimension Budget Allocation for Efficient KV Cache Compression
Ruijie Miao, Zhiming Wang, Wang Li +4
Key-value (KV) caching is widely used to accelerate transformer inference, but its memory cost grows linearly with input length, limiting long-context deployment. Existing token ev…
Rethinking Retrieval-Augmentation as Synthesis: A Query-Aware Context Merging Approach
Jiarui Guo, Yuemeng Xu, Zongwei Lv +6
Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to extend their existing knowledge by dynamically incorporating external information. However, practical d…
MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
Ruijie Miao, Yilun Yao, Zihan Wang +5
The Mixture-of-Experts (MoE) technique has proven to be a promising solution to efficiently scale the model size, which has been widely applied in recent LLM advancements. However,…
Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?
Qinyan Zhang, Xinping Lei, Ruijie Miao +18
Large Language Models (LLMs) achieve strong performance on diverse tasks but often exhibit cognitive inertia, struggling to follow instructions that conflict with the standardized…