Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

Muhammad Mansoor, Tahir Ahmad, Yeo-Chan Yoon

Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not mo…