Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition

Abstract

This paper presents a class of new fast non-trainable entropy-based confidence estimation methods for automatic speech recognition. We show how per-frame entropy values can be normalized and aggregated to obtain a confidence measure per unit and per word for Connectionist Temporal Classification (CTC) and Recurrent Neural Network Transducer (RNN-T) models. Proposed methods have similar computational complexity to the traditional method based on the maximum per-frame probability, but they are more adjustable, have a wider effective threshold range, and better push apart the confidence distributions of correct and incorrect words. We evaluate the proposed confidence measures on LibriSpeech test sets, and show that they are up to 2 and 4 times better than confidence estimation based on the maximum per-frame probability at detecting incorrect words for Conformer-CTC and Conformer-RNN-T models, respectively.

arXiv commentTo appear in Proc. SLT 2022, Jan 09-12, 2023, Doha, Qatar. 8 pages, 4 figures, 4 tables

References in corpus (5)

• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification
• Improved Noisy Student Training for Automatic Speech Recognition
• NeMo: a toolkit for building AI applications using Neural Modules
• Entropy methods for the confidence assessment of probabilistic classification models

Cited by in corpus (1)

• Confidence-based Ensembles of End-to-End Speech Recognition Models