Memory-efficient Stochastic methods for Memory-based Transformers

Vishwajit Kumar Vishnu, C. Chandra Sekhar

Training Memory-based transformers can require a large amount of memory and can be quite inefficient. We propose a novel two-phase training mechanism and a novel regularization tec…