Resource-aware Sub-network Allocation for Heterogeneous Decentralized Learning

Zhuojun Tian, Ming Xiao, Mikael Skoglund

System heterogeneity in computation, communication, and memory can substantially limit synchronous decentralized learning, where each training round is constrained by resource-limi…