Astra: Efficient and Money-saving Automatic Parallel Strategies Search on Heterogeneous GPUs

Peiran Wang, Haibing Li, Fu Haohan +3

In this paper, we introduce an efficient and money-saving automatic parallel strategies search framework on heterogeneous GPUs: Astra. First, Astra searches for the efficiency-opti…