Text-Conditioned Transformer for Automatic Pronunciation Error Detection

Abstract

Automatic pronunciation error detection (APED) plays an important role in the domain of language learning. As for the previous ASR-based APED methods, the decoded results need to be aligned with the target text so that the errors can be found out. However, since the decoding process and the alignment process are independent, the prior knowledge about the target text is not fully utilized. In this paper, we propose to use the target text as an extra condition for the Transformer backbone to handle the APED task. The proposed method can output the error states with consideration of the relationship between the input speech and the target text in a fully end-to-end fashion.Meanwhile, as the prior target text is used as a condition for the decoder input, the Transformer works in a feed-forward manner instead of autoregressive in the inference stage, which can significantly boost the speed in the actual deployment. We set the ASR-based Transformer as the baseline APED model and conduct several experiments on the L2-Arctic dataset. The results demonstrate that our approach can obtain 8.4\% relative improvement on the $F_1$ score metric.

published for Speech Communication journal

References in corpus (7)

• Attention-Based Models for Speech Recognition
• FastSpeech: Fast, Robust and Controllable Text to Speech
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• Scalable Learning of Non-Decomposable Objectives
• Movement Pruning: Adaptive Sparsity by Fine-Tuning
• Non-Autoregressive Neural Text-to-Speech
• A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks

Cited by in corpus (3)

• Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need
• Segmentation-free Goodness of Pronunciation
• Discriminative, Generative and Self-Supervised Approaches for Target-Agnostic Learning