Checkmate: Zero-Overhead Model Checkpointing via Network Gradient Replication

Ankit Bhardwaj, Weiyang Wang, Jeremy Carin +2

This paper presents Checkmate, a system that enables per-iteration checkpointing in DNN training without any training slowdown. The traditional approach to checkpointing requires a…