Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification

Abstract

End-to-end speech-to-intent classification has shown its advantage in harvesting information from both text and speech. In this paper, we study a technique to develop such an end-to-end system that supports multiple languages. To overcome the scarcity of multi-lingual speech corpus, we exploit knowledge from a pre-trained multi-lingual natural language processing model. Multi-lingual bidirectional encoder representations from transformers (mBERT) models are trained on multiple languages and hence expected to perform well in the multi-lingual scenario. In this work, we employ a teacher-student learning approach to sufficiently extract information from an mBERT model to train a multi-lingual speech model. In particular, we use synthesized speech generated from an English-Mandarin text corpus for analysis and training of a multi-lingual intent classification model. We also demonstrate that the teacher-student learning approach obtains an improved performance (91.02%) over the traditional end-to-end (89.40%) intent classification approach in a practical multi-lingual scenario.

References in corpus (5)

• Distilling the Knowledge in a Neural Network
• Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
• THCHS-30 : A Free Chinese Speech Corpus
• Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Model
• Speech-language Pre-training for End-to-end Spoken Language Understanding