Cache Management for Mixture-of-Experts LLMs -- extended version

Spyros Angelopoulos, Loris Marchal, Adrien Obrecht +1

Large language models (LLMs) have demonstrated remarkable capabilities across a variety of tasks. One of the main challenges towards the successful deployment of LLMs is memory man…