7 papers
Retrieval-Augmented Generation for Natural Language Processing: A Survey
Shangyu Wu, Ying Xiong, Yufei Cui +8
Large language models (LLMs) have achieved strong empirical performance in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still…
RAEE: A Robust Retrieval-Augmented Early Exit Framework for Efficient Inference
Lianming Huang, Shangyu Wu, Yufei Cui +6
Deploying large language model inference remains challenging due to their high computational overhead. Early exit optimizes model inference by adaptively reducing the number of inf…
ReFilter: Improving Robustness of Retrieval-Augmented Generation via Gated Filter
Yixin Chen, Ying Xiong, Shangyu Wu +3
Retrieval-augmented generation (RAG) has become a dominant paradigm for grounding large language models (LLMs) with external evidence in knowledge-intensive question answering. A c…
Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever
Yixin Chen, Ying Xiong, Shangyu Wu +4
Tool-augmented large language models (LLMs) leverage external functions to extend their capabilities, but inaccurate function calls can lead to inefficiencies and increased costs.E…
EvoP: Robust LLM Inference via Evolutionary Pruning
Shangyu Wu, Hongchao Du, Ying Xiong +4
Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, but their massive size and computational demands hinder their deployment in reso…
AATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization
Junhui He, Junna Xing, Nan Wang +6
Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache. Retrieval-based…