Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition

Abstract

Contextual knowledge is important for real-world automatic speech recognition (ASR) applications. In this paper, a novel tree-constrained pointer generator (TCPGen) component is proposed that incorporates such knowledge as a list of biasing words into both attention-based encoder-decoder and transducer end-to-end ASR models in a neural-symbolic way. TCPGen structures the biasing words into an efficient prefix tree to serve as its symbolic input and creates a neural shortcut between the tree and the final ASR output distribution to facilitate recognising biasing words during decoding. Systems were trained and evaluated on the Librispeech corpus where biasing words were extracted at the scales of an utterance, a chapter, or a book to simulate different application scenarios. Experimental results showed that TCPGen consistently improved word error rates (WERs) compared to the baselines, and in particular, achieved significant WER reductions on the biasing words. TCPGen is highly efficient: it can handle 5,000 biasing words and distractors and only add a small overhead to memory use and computation cost.

To appear in ASRU 2021

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Attention-Based Models for Speech Recognition
• Improved training of end-to-end attention models for speech recognition
• Exploring Neural Transducers for End-to-End Speech Recognition
• Contextual Language Model Adaptation for Conversational Agents
• Fast and Robust Unsupervised Contextual Biasing for Speech Recognition
• Joint Contextual Modeling for ASR Correction and Language Understanding