GreenServ: Energy-Efficient Context-Aware Dynamic Routing for Multi-Model LLM Inference

Thomas Ziller, Shashikant Ilager, Alessandro Tundo +3

Large language models (LLMs) demonstrate remarkable capabilities, but their broad deployment is limited by significant computational resource demands, particularly energy consumpti…