Leveraging Cross-Utterance Context For ASR Decoding

Abstract

While external language models (LMs) are often incorporated into the decoding stage of automated speech recognition systems, these models usually operate with limited context. Cross utterance information has been shown to be beneficial during second pass re-scoring, however this limits the hypothesis space based on the local information available to the first pass LM. In this work, we investigate the incorporation of long-context transformer LMs for cross-utterance decoding of acoustic models via beam search, and compare against results from n-best rescoring. Results demonstrate that beam search allows for an improved use of cross-utterance context. When evaluating on the long-format dataset AMI, results show a 0.7\% and 0.3\% absolute reduction on dev and test sets compared to the single-utterance setting, with improvements when including up to 500 tokens of prior context. Evaluations are also provided for Tedlium-1 with less significant improvements of around 0.1\% absolute.

References in corpus (14)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• PaLM: Scaling Language Modeling with Pathways
• The Microsoft 2017 Conversational Speech Recognition System
• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
• Language Modeling with Deep Transformers
• CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention
• Character-Level Incremental Speech Recognition with Recurrent Neural Networks
• Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization
• Long-span language modeling for speech recognition
• An Empirical Study of Efficient ASR Rescoring with Transformers
• Cross-utterance Reranking Models with BERT and Graph Convolutional Networks for Conversational Speech Recognition
• Cross-Utterance Language Models with Acoustic Error Sampling
• LSTM-LM with Long-Term History for First-Pass Decoding in Conversational Speech Recognition