22 citations · 43 across the 9 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022★ 8 cited
Survey on Large Scale Neural Network Training
Julia Gusak, Daria Cherniuk, Alena Shilova +8
Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training. Hence, many models do not fit one GPU de…
cs.LG2019★ 13 cited
Optimal checkpointing for heterogeneous chains: how to train deep neural networks with limited memory
Julien Herrmann, Olivier Beaumont, Lionel Eyraud-Dubois +3
This paper introduces a new activation checkpointing method which allows to significantly decrease memory usage when training Deep Neural Networks with the back-propagation algorit…