Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks

Xingchen Song, Guangsen Wang, Zhiyong Wu +4

Self-attention network (SAN) can benefit significantly from the bi-directional representation learning through unsupervised pretraining paradigms such as BERT and XLNet. In this pa…