A Study of Multilingual End-to-End Speech Recognition for Kazakh, Russian, and English

Abstract

We study training a single end-to-end (E2E) automatic speech recognition (ASR) model for three languages used in Kazakhstan: Kazakh, Russian, and English. We first describe the development of multilingual E2E ASR based on Transformer networks and then perform an extensive assessment on the aforementioned languages. We also compare two variants of output grapheme set construction: combined and independent. Furthermore, we evaluate the impact of LMs and data augmentation techniques on the recognition performance of the multilingual E2E ASR. In addition, we present several datasets for training and evaluation purposes. Experiment results show that the multilingual models achieve comparable performances to the monolingual baselines with a similar number of parameters. Our best monolingual and multilingual models achieved 20.9% and 20.5% average word error rates on the combined test set, respectively. To ensure the reproducibility of our experiments and results, we share our training recipes, datasets, and pre-trained models.

arXiv comment12 pages, 3 tables, accepted to SPECOM 2021

References in corpus (5)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• On Using Monolingual Corpora in Neural Machine Translation
• KazakhTTS: An Open-Source Kazakh Text-to-Speech Synthesis Dataset
• Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
• Scaling End-to-End Models for Large-Scale Multilingual ASR