Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod

Ao Xiao, Bangzheng He, Baoquan Zhang +125

Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…