Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Models

Camille Couturier, Spyros Mastorakis, Haiying Shen +2

Large Language Models (LLMs) are increasingly deployed across edge and cloud platforms for real-time question-answering and retrieval-augmented generation. However, processing leng…