A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognition

Abstract

Despite the success of deep learning in speech recognition, multi-dialect speech recognition remains a difficult problem. Although dialect-specific acoustic models are known to perform well in general, they are not easy to maintain when dialect-specific data is scarce and the number of dialects for each language is large. Therefore, a single unified acoustic model (AM) that generalizes well for many dialects has been in demand. In this paper, we propose a novel acoustic modeling technique for accurate multi-dialect speech recognition with a single AM. Our proposed AM is dynamically adapted based on both dialect information and its internal representation, which results in a highly adaptive AM for handling multiple dialects simultaneously. We also propose a simple but effective training method to deal with unseen dialects. The experimental results on large scale speech datasets show that the proposed AM outperforms all the previous ones, reducing word error rates (WERs) by 8.11% relative compared to a single all-dialects AM and by 7.31% relative compared to dialect-specific AMs.

References in corpus (3)

• A Learned Representation For Artistic Style
• Modulating early visual processing by language
• FiLM: Visual Reasoning with a General Conditioning Layer

Cited by in corpus (7)

• Layer-wise Fast Adaptation for End-to-End Multi-Accent Speech Recognition
• The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methods
• Accented Speech Recognition: A Survey
• Improving Accent Identification and Accented Speech Recognition Under a Framework of Self-supervised Learning
• AIPNet: Generative Adversarial Pre-training of Accent-invariant Networks for End-to-end Speech Recognition
• REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling
• Improving Speech Recognition Accuracy of Local POI Using Geographical Models