Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition

Xianrui Zheng, Chao Zhang, Philip C. Woodland

Language models (LMs) pre-trained on massive amounts of text, in particular bidirectional encoder representations from Transformers (BERT), generative pre-training (GPT), and GPT-2…