12 citations · 15 across the 5 of their papers we have counts for
5 papers
ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
Haley Li, Xinglu Wang, Cong Feng +12
As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to ha…
ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
Gursimran Singh, Timothy Yu, Haley Li +7
Mixture-of-Experts (MoE) models promise efficient scaling of large language models (LLMs) by activating only a small subset of experts per token, but their parallelized inference p…
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions
Rindranirina Ramamonjison, Timothy T. Yu, Raymond Li +8
The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text…
Augmenting Operations Research with Auto-Formulation of Optimization Models from Problem Descriptions
Rindranirina Ramamonjison, Haley Li, Timothy T. Yu +5
We describe an augmented intelligence system for simplifying and enhancing the modeling experience for operations research. Using this system, the user receives a suggested formula…