5 papers
Curator: Efficient Vector Search with Low-Selectivity Filters
Yicheng Jin, Yongji Wu, Wenjun Hu +4
Embedding-based dense retrieval has become the cornerstone of many critical applications, where approximate nearest neighbor search (ANNS) queries are often combined with filters o…
Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models
Yongji Wu, Wenjie Qu, Xueshen Liu +10
Sparsely-activated Mixture-of-Experts (MoE) architecture has increasingly been adopted to further scale large language models (LLMs). However, frequent failures still pose signific…
Phantora: Maximizing Code Reuse in Simulation-based Machine Learning System Performance Estimation
Jianxing Qin, Jingrong Chen, Xinhao Kong +8
Modern machine learning (ML) training workloads place substantial demands on both computational and communication resources. Consequently, accurate performance estimation has becom…
Plato: Plan to Efficiently Decode for Large Language Model Inference
Shuowei Jin, Xueshen Liu, Yongji Wu +7
Large language models (LLMs) have achieved remarkable success in natural language tasks, but their inference incurs substantial computational and memory overhead. To improve effici…
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
Yongji Wu, Xueshen Liu, Shuowei Jin +6
The Mixture-of-Experts (MoE) architecture has become increasingly popular as a method to scale up large language models (LLMs). To save costs, heterogeneity-aware training solution…