Knowledge Transfer from Large-scale Pretrained Language Models to End-to-end Speech Recognizers

Yotaro Kubo, Shigeki Karita, Michiel Bacchiani

End-to-end speech recognition is a promising technology for enabling compact automatic speech recognition (ASR) systems since it can unify the acoustic and language model into a si…