Finding consensus in speech recognition: word error minimization and other applications of confusion networks

Abstract

We describe a new framework for distilling information from word lattices to improve the accuracy of speech recognition and obtain a more perspicuous representation of a set of alternative hypotheses. In the standard MAP decoding approach the recognizer outputs the string of words corresponding to the path with the highest posterior probability given the acoustics and a language model. However, even given optimal models, the MAP decoder does not necessarily minimize the commonly used performance metric, word error rate (WER). We describe a method for explicitly minimizing WER by extracting word hypotheses with the highest posterior probabilities from word lattices. We change the standard problem formulation by replacing global search over a large set of sentence hypotheses with local search over a small set of word candidates. In addition to improving the accuracy of the recognizer, our method produces a new representation of the set of candidate hypotheses that specifies the sequence of word-level confusions in a compact lattice format. We study the properties of confusion networks and examine their use for other tasks, such as lattice compression, word spotting, confidence annotation, and reevaluation of recognition hypotheses using higher-level knowledge sources.

35 pages, 8 figures

Cited by in corpus (25)

• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
• Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment
• Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities
• TranSmart: A Practical Interactive Machine Translation System
• End-to-End Spoken Language Understanding: Performance analyses of a voice command task in a low resource setting
• Neural Lattice-to-Sequence Models for Uncertain Inputs
• L2RS: A Learning-to-Rescore Mechanism for Automatic Speech Recognition
• Fast and Accurate OOV Decoder on High-Level Features
• Automatic Quality Estimation for ASR System Combination
• Bootstrapping Lexical Choice via Multiple-Sequence Alignment
• Modeling ASR Ambiguity for Dialogue State Tracking Using Word Confusion Networks
• Multi-Graph Decoding for Code-Switching ASR
• ASR error management for improving spoken language understanding
• Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognition
• Learning Word-Level Confidence For Subword End-to-End ASR
• Utilisation de la linguistique en reconnaissance de la parole : un état de l'art
• On Addressing Practical Challenges for RNN-Transducer
• Exploring Gaussian mixture model framework for speaker adaptation of deep neural network acoustic models
• Towards Better Understanding of Spontaneous Conversations: Overcoming Automatic Speech Recognition Errors With Intent Recognition
• Streaming Speech-to-Confusion Network Speech Recognition
• Semi-tied Units for Efficient Gating in LSTM and Highway Networks
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation
• Word-level confidence estimation for RNN transducers
• On combining features for single-channel robust speech recognition in reverberant environments
• DNN adaptation by automatic quality estimation of ASR hypotheses