End-to-End Spoken Language Translation

Abstract

In this paper, we address the task of spoken language understanding. We present a method for translating spoken sentences from one language into spoken sentences in another language. Given spectrogram-spectrogram pairs, our model can be trained completely from scratch to translate unseen sentences. Our method consists of a pyramidal-bidirectional recurrent network combined with a convolutional network to output sentence-level spectrograms in the target language. Empirically, our model achieves competitive performance with state-of-the-art methods on multiple languages and can generalize to unseen speakers.

arXiv commentTechnical Report. Stanford University, 2017. arXiv admin note: text overlap with arXiv:1804.00047

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Attention-Based Models for Speech Recognition
• Deep Voice: Real-time Neural Text-to-Speech
• Dialogue Learning With Human-In-The-Loop

Cited by in corpus (3)

• Direct speech-to-speech translation with a sequence-to-sequence model
• Unsupervised Learning of Audio Perception for Robotics Applications: Learning to Project Data to T-SNE/UMAP space
• Learning to Model Aspects of Hearing Perception Using Neural Loss Functions