CALM: A Self-Adaptive Orchestration Approach for QoS-Aware Routing in Small Language Model based Systems

Hemang Jain, Divyansh Pandey, Karthik Vaidhyanathan

AI-enabled systems are subjected to various types of runtime uncertainties, ranging from dynamic workloads, resource requirements, model drift, etc. These uncertainties have a big…