Dynamic Tensor Rematerialization

Marisa Kirisame, Steven Lyubomirsky, Altan Haan +5

Checkpointing enables the training of deep learning models under restricted memory budgets by freeing intermediate activations from memory and recomputing them on demand. Current c…