Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection

Yun Zhu, Jia-Chen Gu, Caitlin Sikora +8

Large language models (LLMs) augmented with retrieval exhibit robust performance and extensive versatility by incorporating external contexts. However, the input length grows linea…