LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters

Reza Farahani, Zoha Azimi, Mario Colosi +1

Large language model (LLM) services increasingly operate on edge infrastructure, enabling low-latency and privacy-preserving AI services. However, efficiently serving LLM requests…