Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

Cunchen Hu, Liangliang Xu, Tian Liu +9

Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing ener…