Distilling the Knowledge of BERT for Sequence-to-Sequence ASR

Hayato Futami, Hirofumi Inaguma, Sei Ueno +3

Attention-based sequence-to-sequence (seq2seq) models have achieved promising results in automatic speech recognition (ASR). However, as these models decode in a left-to-right way,…