Rise of Distributed Deep Learning Training in the Big Model Era: From a Software Engineering Perspective

Xuanzhe Liu, Diandian Gu, Zhenpeng Chen +5

Deep learning (DL) has become a key component of modern software. In the "big model" era, the rich features of DL-based software substantially rely on powerful DL models, e.g., BER…