4 papers
CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
Adrian Zhao, Zhenkun Cai, Zhenyu Song +5
Mixture-of-Experts (MoE) has recently emerged as the mainstream architecture for efficiently scaling large language models while maintaining near-constant computational cost. Exper…
DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
Chenyu Jiang, Zhenkun Cai, Ye Tian +3
Context parallelism has emerged as a key technique to support long-context training, a growing trend in generative AI for modern large models. However, existing context parallel me…
MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy
Haitian Jiang, Renjie Liu, Zengfeng Huang +5
Among the many variants of graph neural network (GNN) architectures capable of modeling data with cross-instance relations, an important subclass involves layers designed such that…
DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Training
Renjie Liu, Yichuan Wang, Xiao Yan +5
Graph neural networks (GNNs) are machine learning models specialized for graph data and widely used in many applications. To train GNNs on large graphs that exceed CPU memory, seve…