Training Time Prediction for Mixed Precision-based Distributed Training

Minchul Kang, Changyong Shin, Jinwoo Jeong +5

Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precis…