Distilling a Pretrained Language Model to a Multilingual ASR Model

Kwanghee Choi, Hyung-Min Park

Multilingual speech data often suffer from long-tailed language distribution, resulting in performance degradation. However, multilingual text data is much easier to obtain, yieldi…