11 citations · 16 across the 10 of their papers we have counts for
10 papers
ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
Haley Li, Xinglu Wang, Cong Feng +12
As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to ha…
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
LLMPrism: Black-box Performance Diagnosis for Production LLM Training Platforms
Zhihan Jiang, Rui Ren, Guangba Yu +8
Large Language Models (LLMs) have brought about revolutionary changes in diverse fields, rendering LLM training of utmost importance for modern enterprises. To meet this demand, mu…
L4: Diagnosing Large-scale LLM Training Failures via Automated Log Analysis
Zhihan Jiang, Junjie Huang, Zhuangbin Chen +6
As Large Language Models (LLMs) show their capabilities across various applications, training customized LLMs has become essential for modern enterprises. However, due to the compl…
Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis
Junjie Huang, Zhihan Jiang, Jinyang Liu +7
Logs are imperative in the maintenance of online service systems, which often encompass important information for effective failure mitigation. While existing anomaly detection met…
FaultProfIT: Hierarchical Fault Profiling of Incident Tickets in Large-scale Cloud Systems
Junjie Huang, Jinyang Liu, Zhuangbin Chen +7
Postmortem analysis is essential in the management of incidents within cloud systems, which provides valuable insights to improve system's reliability and robustness. At CloudA, fa…
Practical Anomaly Detection over Multivariate Monitoring Metrics for Online Services
Jinyang Liu, Tianyi Yang, Zhuangbin Chen +4
As modern software systems continue to grow in terms of complexity and volume, anomaly detection on multivariate monitoring metrics, which profile systems' health status, becomes m…
Prism: Revealing Hidden Functional Clusters from Massive Instances in Cloud Systems
Jinyang Liu, Zhihan Jiang, Jiazhen Gu +6
Ensuring the reliability of cloud systems is critical for both cloud vendors and customers. Cloud systems often rely on virtualization techniques to create instances of hardware re…
Identifying Performance Issues in Cloud Service Systems Based on Relational-Temporal Features
Wenwei Gu, Jinyang Liu, Zhuangbin Chen +7
Cloud systems are susceptible to performance issues, which may cause service-level agreement violations and financial losses. In current practice, crucial metrics are monitored per…
Log-based Anomaly Detection based on EVT Theory with feedback
Jinyang Liu, Junjie Huang, Yintong Huo +6
System logs play a critical role in maintaining the reliability of software systems. Fruitful studies have explored automatic log-based anomaly detection and achieved notable accur…