Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition

Abstract

Unsupervised domain adaptation of speech signal aims at adapting a well-trained source-domain acoustic model to the unlabeled data from target domain. This can be achieved by adversarial training of deep neural network (DNN) acoustic models to learn an intermediate deep representation that is both senone-discriminative and domain-invariant. Specifically, the DNN is trained to jointly optimize the primary task of senone classification and the secondary task of domain classification with adversarial objective functions. In this work, instead of only focusing on learning a domain-invariant feature (i.e. the shared component between domains), we also characterize the difference between the source and target domain distributions by explicitly modeling the private component of each domain through a private component extractor DNN. The private component is trained to be orthogonal with the shared component and thus implicitly increases the degree of domain-invariance of the shared component. A reconstructor DNN is used to reconstruct the original speech feature from the private and shared components as a regularization. This domain separation framework is applied to the unsupervised environment adaptation task and achieved 11.08% relative WER reduction from the gradient reversal layer training, a representative adversarial training method, for automatic speech recognition on CHiME-3 dataset.

8 pages, 1 figure, ASRU 2017

References in corpus (5)

• Unsupervised Domain Adaptation by Backpropagation
• Domain Separation Networks
• Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition
• Invariant Representations for Noisy Speech Recognition
• Large-Scale Domain Adaptation via Teacher-Student Learning

Cited by in corpus (26)

• Speaker-Invariant Training via Adversarial Learning
• Conditional Teacher-Student Learning
• Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
• Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
• Adversarial Speaker Verification
• Cycle-Consistent Speech Enhancement
• Adversarial Feature-Mapping for Speech Enhancement
• Speaker Adaptation for Attention-Based End-to-End Speech Recognition
• Adversarial Speaker Adaptation
• Semi-Supervised Speech Recognition via Local Prior Matching
• Noise Adaptive Speech Enhancement using Domain Adversarial Training
• Towards Unsupervised Automatic Speech Recognition Trained by Unaligned Speech and Text only
• Unsupervised Domain Adaptation by Adversarial Learning for Robust Speech Recognition
• Attentive Adversarial Learning for Domain-Invariant Training
• Enhancing and Adversarial: Improve ASR with Speaker Labels
• Unsupervised Domain Adaptation Schemes for Building ASR in Low-resource Languages
• Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training
• The CUHK-TUDELFT System for The SLT 2021 Children Speech Recognition Challenge
• Recent Progresses in Deep Learning based Acoustic Models (Updated)
• Active Voice Authentication
• Dynamic Layer Customization for Noise Robust Speech Emotion Recognition in Heterogeneous Condition Training
• Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
• L-Vector: Neural Label Embedding for Domain Adaptation
• Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition
• Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
• Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval