HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters

Antian Liang, Zhigang Zhao, Kai Zhang +6

With the rapid evolution of GPU architectures, the heterogeneity of model training infrastructures is steadily increasing. In such environments, effectively utilizing all available…