QAM: Quadratic-Accurate Checkpoint Merging via Sequential Consistency

Shihao Wang, Rui Kong, Xinran Chen +7

Saved checkpoints record states along a training trajectory, but generally do not determine the updates at states that would be visited under a different schedule. We study how acc…