A Survey of End-to-End Modeling for Distributed DNN Training: Workloads, Simulators, and TCO

Jonas Svedas, Hannah Watson, Nathan Laubeuf +6

Distributed deep neural networks (DNNs) have become a cornerstone for scaling machine learning to meet the demands of increasingly complex applications. However, the rapid growth i…