Highly Available Data Parallel ML training on Mesh Networks

Sameer Kumar, Norm Jouppi

Data parallel ML models can take several days or weeks to train on several accelerators. The long duration of training relies on the cluster of resources to be available for the jo…