Leveraging Approximate Caching for Faster Retrieval-Augmented Generation

Shai Bergman, Anne-Marie Kermarrec, Diana Petrescu +4

Retrieval-augmented generation (RAG) improves the reliability of large language model (LLM) answers by integrating external knowledge. However, RAG increases the end-to-end inferen…