Efficiently Train ASR Models that Memorize Less and Perform Better with Per-core Clipping

Lun Wang, Om Thakkar, Zhong Meng +3

Gradient clipping plays a vital role in training large-scale automatic speech recognition (ASR) models. It is typically applied to minibatch gradients to prevent gradient explosion…