A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognition

Abstract

Wav2vec2.0 is a popular self-supervised pre-training framework for learning speech representations in the context of automatic speech recognition (ASR). It was shown that wav2vec2.0 has a good robustness against the domain shift, while the noise robustness is still unclear. In this work, we therefore first analyze the noise robustness of wav2vec2.0 via experiments. We observe that wav2vec2.0 pre-trained on noisy data can obtain good representations and thus improve the ASR performance on the noisy test set, which however brings a performance degradation on the clean test set. To avoid this issue, in this work we propose an enhanced wav2vec2.0 model. Specifically, the noisy speech and the corresponding clean version are fed into the same feature encoder, where the clean speech provides training targets for the model. Experimental results reveal that the proposed method can not only improve the ASR performance on the noisy test set which surpasses the original wav2vec2.0, but also ensure a tiny performance decrease on the clean test set. In addition, the effectiveness of the proposed method is demonstrated under different types of noise conditions.

arXiv commentAccepted by ICASSP 2022

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Representation Learning with Contrastive Predictive Coding
• Categorical Reparameterization with Gumbel-Softmax
• Unsupervised Domain Adaptation by Backpropagation
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
• TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition
• Improving Transformer-based Speech Recognition Using Unsupervised Pre-training
• Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstruction

Cited by in corpus (5)

• Self-Supervised Speech Representation Learning: A Review
• Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers
• Dual-Branch Knowledge Distillation for Noise-Robust Synthetic Speech Detection
• CPT-Boosted Wav2vec2.0: Towards Noise Robust Speech Recognition for Classroom Environments
• WavRx: a Disease-Agnostic, Generalizable, and Privacy-Preserving Speech Health Diagnostic Model