SYMPHONY: Improving Memory Management for LLM Inference Workloads

Saurabh Agarwal, Anyong Mao, Aditya Akella +1

Large Language Models (LLMs) are increasingly being deployed in applications such as chatbots, code editors, and conversational agents. A key feature of LLMs is their ability to en…