RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding

Guanzheng Chen, Qilong Feng, Jinjie Ni +2

The emergence of long-context large language models (LLMs) offers a promising alternative to traditional retrieval-augmented generation (RAG) for processing extensive documents. Ho…