Deep context: end-to-end contextual speech recognition

Abstract

In automatic speech recognition (ASR) what a user says depends on the particular context she is in. Typically, this context is represented as a set of word n-grams. In this work, we present a novel, all-neural, end-to-end (E2E) ASR sys- tem that utilizes such context. Our approach, which we re- fer to as Contextual Listen, Attend and Spell (CLAS) jointly- optimizes the ASR components along with embeddings of the context n-grams. During inference, the CLAS system can be presented with context phrases which might contain out-of- vocabulary (OOV) terms not seen during training. We com- pare our proposed system to a more traditional contextualiza- tion approach, which performs shallow-fusion between inde- pendently trained LAS and contextual n-gram models during beam search. Across a number of tasks, we find that the pro- posed CLAS system outperforms the baseline method by as much as 68% relative WER, indicating the advantage of joint optimization over individually trained components. Index Terms: speech recognition, sequence-to-sequence models, listen attend and spell, LAS, attention, embedded speech recognition.

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• Sequence Transduction with Recurrent Neural Networks
• Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
• Personalized Speech recognition on mobile devices
• Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models

Cited by in corpus (6)

• Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models
• Cross-Attention End-to-End ASR for Two-Party Conversations
• Neural Composition: Learning to Generate from Multiple Models
• Contextual Speech Recognition with Difficult Negative Training Examples
• End-to-end contextual speech recognition using class language models and a token passing decoder
• Acoustic-to-Word Models with Conversational Context Information