3 papers
cs.DC2025
MoFa: A Unified Performance Modeling Framework for LLM Pretraining
Lu Zhao, Rong Shi, Shaoqing Zhang +14
The exponential growth in LLM scales, with parameters soaring from billions to trillions, has necessitated distributed pretraining across large clusters comprising thousands to ten…
cs.LG2025
AI-Integrated Decision Support System for Real-Time Market Growth Forecasting and Multi-Source Content Diffusion Analytics
Ziqing Yin, Xuanjing Chen, Xi Zhang
The rapid proliferation of AI-generated content (AIGC) has reshaped the dynamics of digital marketing and online consumer behavior. However, predicting the diffusion trajectory and…
cs.DC2025
MemFine: Memory-Aware Fine-Grained Scheduling for MoE Training
Lu Zhao, Rong Shi, Shaoqing Zhang +21
The training of large-scale Mixture of Experts (MoE) models faces a critical memory bottleneck due to severe load imbalance caused by dynamic token routing. This imbalance leads to…