Adversarial Speaker Adaptation

Abstract

We propose a novel adversarial speaker adaptation (ASA) scheme, in which adversarial learning is applied to regularize the distribution of deep hidden features in a speaker-dependent (SD) deep neural network (DNN) acoustic model to be close to that of a fixed speaker-independent (SI) DNN acoustic model during adaptation. An additional discriminator network is introduced to distinguish the deep features generated by the SD model from those produced by the SI model. In ASA, with a fixed SI model as the reference, an SD model is jointly optimized with the discriminator network to minimize the senone classification loss, and simultaneously to mini-maximize the SI/SD discrimination loss on the adaptation data. With ASA, a senone-discriminative deep feature is learned in the SD model with a similar distribution to that of the SI model. With such a regularized and adapted deep feature, the SD model can perform improved automatic speech recognition on the target speaker's speech. Evaluated on the Microsoft short message dictation dataset, ASA achieves 14.4% and 7.9% relative word error rate improvements for supervised and unsupervised adaptation, respectively, over an SI model trained from 2600 hours data, with 200 adaptation utterances per speaker.

5 pages, 2 figures, ICASSP 2019

References in corpus (12)

• Unsupervised Domain Adaptation by Backpropagation
• Domain Separation Networks
• Speaker-Invariant Training via Adversarial Learning
• Conditional Teacher-Student Learning
• Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition
• Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
• Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
• Invariant Representations for Noisy Speech Recognition
• Adversarial Speaker Verification
• Cycle-Consistent Speech Enhancement
• Adversarial Feature-Mapping for Speech Enhancement
• Attentive Adversarial Learning for Domain-Invariant Training

Cited by in corpus (13)

• Conditional Teacher-Student Learning
• Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• Adversarial Speaker Verification
• Speaker Adaptation for Attention-Based End-to-End Speech Recognition
• Attentive Adversarial Learning for Domain-Invariant Training
• L-Vector: Neural Label Embedding for Domain Adaptation
• Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
• Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition
• A Unified Speaker Adaptation Approach for ASR
• Attention-based gated scaling adaptative acoustic model for ctc-based speech recognition
• Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
• Optimal Transport-based Adaptation in Dysarthric Speech Tasks