Invariant Representations for Noisy Speech Recognition

Abstract

Modern automatic speech recognition (ASR) systems need to be robust under acoustic variability arising from environmental, speaker, channel, and recording conditions. Ensuring such robustness to variability is a challenge in modern day neural network-based ASR systems, especially when all types of variability are not seen during training. We attempt to address this problem by encouraging the neural network acoustic model to learn invariant feature representations. We use ideas from recent research on image generation using Generative Adversarial Networks and domain adaptation ideas extending adversarial gradient-based training. A recent work from Ganin et al. proposes to use adversarial training for image domain adaptation by using an intermediate representation from the main target classification network to deteriorate the domain classifier performance through a separate neural network. Our work focuses on investigating neural architectures which produce representations invariant to noise conditions for ASR. We evaluate the proposed architecture on the Aurora-4 task, a popular benchmark for noise robust ASR. We show that our method generalizes better than the standard multi-condition training especially when only a few noise categories are seen during training.

5 pages, 1 figure, 1 table, NIPS workshop on end-to-end speech recognition

References in corpus (3)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Feature Learning in Deep Neural Networks - Studies on Speech Recognition Tasks
• Blocks and Fuel: Frameworks for deep learning

Cited by in corpus (27)

• Speaker-Invariant Training via Adversarial Learning
• Characterizing Audio Adversarial Examples Using Temporal Dependency
• Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
• Generalizing to unseen domains via distribution matching
• Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• Adversarial Speaker Verification
• Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers
• Learning Independent Features with Adversarial Nets for Non-linear ICA
• Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?
• Adversarial Feature-Mapping for Speech Enhancement
• Speaker Adaptation for Attention-Based End-to-End Speech Recognition
• Adversarial Speaker Adaptation
• Adversarial Network Bottleneck Features for Noise Robust Speaker Verification
• Noise Adaptive Speech Enhancement using Domain Adversarial Training
• Domain Adversarial Training for Accented Speech Recognition
• Unsupervised Domain Adaptation by Adversarial Learning for Robust Speech Recognition
• Attentive Adversarial Learning for Domain-Invariant Training
• Enhancing and Adversarial: Improve ASR with Speaker Labels
• Speaker De-identification System using Autoencoders and Adversarial Training
• Learning Noise-Invariant Representations for Robust Speech Recognition
• AIPNet: Generative Adversarial Pre-training of Accent-invariant Networks for End-to-end Speech Recognition
• Recent Progresses in Deep Learning based Acoustic Models (Updated)
• Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
• What does a network layer hear? Analyzing hidden representations of end-to-end ASR through speech synthesis
• Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
• GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL