MemBoost: A Memory-Boosted Framework for Cost-Aware LLM Inference

Joris Köster, Zixuan Liu, Siavash Khajavi +1

Large Language Models (LLMs) deliver strong performance but incur high inference cost in real-world services, especially under workloads with repeated or near-duplicate queries acr…