Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models

Xiaoyu Yang, Qiujia Li, Philip C. Woodland

Self-supervised pre-training is an effective approach to leveraging a large amount of unlabelled data to reduce word error rates (WERs) of automatic speech recognition (ASR) system…