Cross-lingual Embedding Clustering for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition

Abstract

We present a novel approach centered on the decoding stage of Automatic Speech Recognition (ASR) that enhances multilingual performance, especially for low-resource languages. It utilizes a cross-lingual embedding clustering method to construct a hierarchical Softmax (H-Softmax) decoder, which enables similar tokens across different languages to share similar decoder representations. It addresses the limitations of the previous Huffman-based H-Softmax method, which relied on shallow features in token similarity assessments. Through experiments on a downsampled dataset of 15 languages, we demonstrate the effectiveness of our approach in improving low-resource multilingual ASR accuracy.

Published in IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)

References in corpus (4)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• Effectiveness of Hierarchical Softmax in Large Scale Classification Tasks
• Cross-lingual Embedding Clustering for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition

Cited by in corpus (1)

• Cross-lingual Embedding Clustering for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition