4 papers · 1 filter
ReFilter: Improving Robustness of Retrieval-Augmented Generation via Gated Filter
Yixin Chen, Ying Xiong, Shangyu Wu +3
Retrieval-augmented generation (RAG) has become a dominant paradigm for grounding large language models (LLMs) with external evidence in knowledge-intensive question answering. A c…
EvoP: Robust LLM Inference via Evolutionary Pruning
Shangyu Wu, Hongchao Du, Ying Xiong +4
Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, but their massive size and computational demands hinder their deployment in reso…
AATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization
Junhui He, Junna Xing, Nan Wang +6
Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache. Retrieval-based…
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification
Junhui He, Shangyu Wu, Weidong Wen +2
Deploying large language models (LLMs) on edge devices presents significant challenges due to the substantial computational overhead and memory requirements. Activation sparsificat…