An Ensemble Embedding Approach for Improving Semantic Caching Performance in LLM-based Systems

Shervin Ghaffari, Zohre Bahranifard, Mohammad Akbari

Semantic caching enhances the efficiency of large language model (LLM) systems by identifying semantically similar queries, storing responses once, and serving them for subsequent…