KL Regularized Normalization Framework for Low Resource Tasks

Neeraj Kumar, Ankur Narang, Brejesh Lall

Large pre-trained models, such as Bert, GPT, and Wav2Vec, have demonstrated great potential for learning representations that are transferable to a wide variety of downstream tasks…