Cross-Attention End-to-End ASR for Two-Party Conversations
arXiv:1907.10726
Abstract
We present an end-to-end speech recognition model that learns interaction between two speakers based on the turn-changing information. Unlike conventional speech recognition models, our model exploits two speakers' history of conversational-context information that spans across multiple turns within an end-to-end framework. Specifically, we propose a speaker-specific cross-attention mechanism that can look at the output of the other speaker side as well as the one of the current speaker for better at recognizing long conversations. We evaluated the models on the Switchboard conversational speech corpus and show that our model outperforms standard end-to-end speech recognition models.
arXiv commentInterspeech 2019
References in corpus (17)
- Sequence to Sequence Learning with Neural Networks
- ADADELTA: An Adaptive Learning Rate Method
- On the difficulty of training Recurrent Neural Networks
- Attention-Based Models for Speech Recognition
- Machine Comprehension Using Match-LSTM and Answer Pointer
- End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
- Listen, Attend and Spell
- Improved training of end-to-end attention models for speech recognition
- Advances in All-Neural Speech Recognition
- ESPnet: End-to-End Speech Processing Toolkit
- Document Context Language Models
- Learning Natural Language Inference with LSTM
- Larger-Context Language Modelling
- Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion
- Deep context: end-to-end contextual speech recognition
- Acoustic-to-Word Recognition with Sequence-to-Sequence Models
- Contextual Speech Recognition with Difficult Negative Training Examples