2 papers
cs.LG2024
Communication-Efficient Training Workload Balancing for Decentralized Multi-Agent Learning
Seyed Mahmoud Sajjadi Mohammadabadi, Lei Yang, Feng Yan +1
Decentralized Multi-agent Learning (DML) enables collaborative model training while preserving data privacy. However, inherent heterogeneity in agents' resources (computation, comm…
cs.DC2024
MalleTrain: Deep Neural Network Training on Unfillable Supercomputer Nodes
Xiaolong Ma, Feng Yan, Lei Yang +4
First-come first-serve scheduling can result in substantial (up to 10%) of transiently idle nodes on supercomputers. Recognizing that such unfilled nodes are well-suited for deep n…