Recurrent Graph Syntax Encoder for Neural Machine Translation
arXiv:1908.06559
Abstract
Syntax-incorporated machine translation models have been proven successful in improving the model's reasoning and meaning preservation ability. In this paper, we propose a simple yet effective graph-structured encoder, the Recurrent Graph Syntax Encoder, dubbed \textbf{RGSE}, which enhances the ability to capture useful syntactic information. The RGSE is done over a standard encoder (recurrent or self-attention encoder), regarding recurrent network units as graph nodes and injects syntactic dependencies as edges, such that RGSE models syntactic dependencies and sequential information (\textit{i.e.}, word order) simultaneously. Our approach achieves considerable improvements over several syntax-aware NMT models in EnglishGerman and EnglishCzech translation tasks. And RGSE-equipped big model obtains competitive result compared with the state-of-the-art model in WMT14 En-De task. Extensive analysis further verifies that RGSE could benefit long sentence modeling, and produces better translations.
Work in Progress
References in corpus (12)
- Sequence to Sequence Learning with Neural Networks
- Semi-Supervised Classification with Graph Convolutional Networks
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Inductive Representation Learning on Large Graphs
- Convolutional Sequence to Sequence Learning
- Pay Less Attention with Lightweight and Dynamic Convolutions
- Weighted Transformer Network for Machine Translation
- Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
- Modeling Source Syntax for Neural Machine Translation
- Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning
- N-ary Relation Extraction using Graph State LSTM
- Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations