CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognition

Rupak Vignesh Swaminathan, Brian King, Grant P. Strimel +2

We propose a simple yet effective method to compress an RNN-Transducer (RNN-T) through the well-known knowledge distillation paradigm. We show that the transducer's encoder outputs…