Acoustic-to-Word Models with Conversational Context Information

Abstract

Conversational context information, higher-level knowledge that spans across sentences, can help to recognize a long conversation. However, existing speech recognition models are typically built at a sentence level, and thus it may not capture important conversational context information. The recent progress in end-to-end speech recognition enables integrating context with other available information (e.g., acoustic, linguistic resources) and directly recognizing words from speech. In this work, we present a direct acoustic-to-word, end-to-end speech recognition model capable of utilizing the conversational context to better process long conversations. We evaluate our proposed approach on the Switchboard conversational speech corpus and show that our system outperforms a standard end-to-end speech recognition system.

arXiv commentNAACL 2019. arXiv admin note: text overlap with arXiv:1808.02171

References in corpus (20)

• Sequence to Sequence Learning with Neural Networks
• ADADELTA: An Adaptive Learning Rate Method
• On the difficulty of training Recurrent Neural Networks
• Attention-Based Models for Speech Recognition
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• The Microsoft 2016 Conversational Speech Recognition System
• Listen, Attend and Spell
• Improved training of end-to-end attention models for speech recognition
• Advances in All-Neural Speech Recognition
• ESPnet: End-to-End Speech Processing Toolkit
• Document Context Language Models
• Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
• Larger-Context Language Modelling
• Direct Acoustics-to-Word Models for English Conversational Speech Recognition
• Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM
• Deep context: end-to-end contextual speech recognition
• Advancing Acoustic-to-Word CTC Model
• Acoustic-to-Word Recognition with Sequence-to-Sequence Models
• Multi-Modal Data Augmentation for End-to-End ASR
• Dialog-context aware end-to-end speech recognition