Publications (5)
Augmenting Operations Research with Auto-Formulation of Optimization Models from Problem Descriptions
Rindranirina Ramamonjison, Haley Li, Timothy T. Yu +5
We describe an augmented intelligence system for simplifying and enhancing the modeling experience for operations research. Using this system, the user receives a suggested formula…
NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions
Rindranirina Ramamonjison, Timothy T. Yu, Raymond Li +8
The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text…
ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
Haley Li, Xinglu Wang, Cong Feng +12
As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to ha…
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
Gursimran Singh, Timothy Yu, Haley Li +7
Mixture-of-Experts (MoE) models promise efficient scaling of large language models (LLMs) by activating only a small subset of experts per token, but their parallelized inference p…