Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstruction

Weiran Wang, Qingming Tang, Karen Livescu

We propose an approach for pre-training speech representations via a masked reconstruction loss. Our pre-trained encoder networks are bidirectional and can therefore be used direct…