MaaSO: SLO-aware Orchestration of Heterogeneous Model Instances for MaaS

Mo Xuan, Zhang yue, Wu Weigang

Model-as-a-Service (MaaS) platforms face diverse Service Level Objective (SLO) requirements stemming from various large language model (LLM) applications, manifested in contextual…