Lattice Transformer for Speech Translation
arXiv:1906.05551
Abstract
Recent advances in sequence modeling have highlighted the strengths of the transformer architecture, especially in achieving state-of-the-art machine translation results. However, depending on the up-stream systems, e.g., speech recognition, or word segmentation, the input to translation system can vary greatly. The goal of this work is to extend the attention mechanism of the transformer to naturally consume the lattice in addition to the traditional sequential input. We first propose a general lattice transformer for speech translation where the input is the output of the automatic speech recognition (ASR) which contains multiple paths and posterior scores. To leverage the extra information from the lattice structure, we develop a novel controllable lattice attention mechanism to obtain latent representations. On the LDC Spanish-English speech translation corpus, our experiments show that lattice transformer generalizes significantly better and outperforms both a transformer baseline and a lattice LSTM. Additionally, we validate our approach on the WMT 2017 Chinese-English translation task with lattice inputs from different BPE segmentations. In this task, we also observe the improvements over strong baselines.
arXiv commentaccepted to ACL 2019
References in corpus (8)
- Sequence to Sequence Learning with Neural Networks
- Achieving Human Parity on Automatic Chinese to English News Translation
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
- Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation
- Unsupervised Multi-modal Neural Machine Translation
- Regularizing Neural Machine Translation by Target-bidirectional Agreement
- "Bilingual Expert" Can Find Translation Errors