SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving

Andreas Kosmas Kakolyris, Dimosthenis Masouros, Petros Vavaroutsos +2

As Large Language Models (LLMs) gain traction, their reliance on power-hungry GPUs places ever-increasing energy demands, raising environmental and monetary concerns. Inference dom…