Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Memory-adaptive Depth-wise Heterogeneous Federated Learning
Kai Zhang, Yutong Dai, Hongyi Wang +3
Federated learning is a promising paradigm that allows multiple clients to collaboratively train a model without sharing the local data. However, the presence of heterogeneous devi…
cs.LG2024
On Optimizing the Communication of Model Parallelism
Yonghao Zhuang, Hexu Zhao, Lianmin Zheng +6
We study a novel and important communication pattern in large-scale model-parallel deep learning (DL), which we call cross-mesh resharding. This pattern emerges when the two paradi…