Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

Abstract

We present a structured overview of adaptation algorithms for neural network-based speech recognition, considering both hybrid hidden Markov model / neural network systems and end-to-end neural network systems, with a focus on speaker adaptation, domain adaptation, and accent adaptation. The overview characterizes adaptation algorithms as based on embeddings, model parameter adaptation, or data augmentation. We present a meta-analysis of the performance of speech recognition adaptation algorithms, based on relative error rate reductions as reported in the literature.

arXiv commentTotal of 31 pages, 27 figures. Associated repository: https://github.com/pswietojanski/ojsp_adaptation_review_2020

References in corpus (44)

• Distilling the Knowledge in a Neural Network
• Prototypical Networks for Few-shot Learning
• Unsupervised Domain Adaptation by Backpropagation
• Attention-Based Models for Speech Recognition
• Variational Inference with Normalizing Flows
• Sequence Transduction with Recurrent Neural Networks
• A review of domain adaptation without target labels
• On Using Monolingual Corpora in Neural Machine Translation
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• Normalizing Flows for Probabilistic Modeling and Inference
• Improved training of end-to-end attention models for speech recognition
• AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale
• EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
• First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
• Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation
• Conditional Teacher-Student Learning
• Improving Neural Language Models with a Continuous Cache
• Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
• Exploring Neural Transducers for End-to-End Speech Recognition
• Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
• Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
• Dynamic Evaluation of Neural Sequence Models
• Cycle-Consistent Speech Enhancement
• Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion
• Augmented Cyclic Adversarial Learning for Low Resource Domain Adaptation
• Speaker Adaptation for Attention-Based End-to-End Speech Recognition
• Adversarial Speaker Adaptation
• A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognition
• Leveraging native language information for improved accented speech recognition
• Cross-Language Transfer Learning, Continuous Learning, and Domain Adaptation for End-to-End Automatic Speech Recognition
• Semi-Supervised Model Training for Unbounded Conversational Speech Recognition
• Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition
• Unsupervised Domain Adaptation by Adversarial Learning for Robust Speech Recognition
• Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model
• Differentiable Pooling for Unsupervised Acoustic Model Adaptation
• Lattice-Based Unsupervised Test-Time Adaptation of Neural Network Acoustic Models
• Embedding-Based Speaker Adaptive Training of Deep Neural Networks
• Latent Dirichlet Allocation Based Organisation of Broadcast Media Archives for Deep Neural Network Adaptation
• DARTS: Dialectal Arabic Transcription System
• Improving RNN Transducer Modeling for End-to-End Speech Recognition
• Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR
• A Multi-Discriminator CycleGAN for Unsupervised Non-Parallel Speech Domain Adaptation
• Advancing Multi-Accented LSTM-CTC Speech Recognition using a Domain Specific Student-Teacher Learning Paradigm
• Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entities

Cited by in corpus (9)

• Self-Supervised Speech Representation Learning: A Review
• Generalizing Speaker Verification for Spoof Awareness in the Embedding Space
• A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Data
• On Addressing Practical Challenges for RNN-Transducer
• TS-Net: OCR Trained to Switch Between Text Transcription Styles
• Understanding Personalization Gains in Whisper ASR
• Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
• Efficient Extraction of Noise-Robust Discrete Units from Self-Supervised Speech Models
• Seed Words Based Data Selection for Language Model Adaptation