21 citations · 26 across the 8 of their papers we have counts for
8 papers
Learning Hard Retrieval Decoder Attention for Transformers
Hongfei Xu, Qiuhui Liu, Josef van Genabith +1
The Transformer translation model is based on the multi-head attention mechanism, which can be parallelized easily. The multi-head attention network performs the scaled dot-product…
Rewiring the Transformer with Depth-Wise LSTMs
Hongfei Xu, Yang Song, Qiuhui Liu +2
Stacking non-linear layers allows deep neural networks to model complicated functions, and including residual connections in Transformer layers is beneficial for convergence and pe…
Learning Source Phrase Representations for Neural Machine Translation
Hongfei Xu, Josef van Genabith, Deyi Xiong +2
The Transformer translation model (Vaswani et al., 2017) based on a multi-head attention mechanism can be computed effectively in parallel and has significantly pushed forward the…
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change
Hongfei Xu, Josef van Genabith, Deyi Xiong +1
The choice of hyper-parameters affects the performance of neural models. While much previous research (Sutskever et al., 2013; Duchi et al., 2011; Kingma and Ba, 2015) focuses on a…
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers
Hongfei Xu, Josef van Genabith, Qiuhui Liu +1
Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches. Previous work focuses…
Lipschitz Constrained Parameter Initialization for Deep Transformers
Hongfei Xu, Qiuhui Liu, Josef van Genabith +2
The Transformer translation model employs residual connection and layer normalization to ease the optimization difficulties caused by its multi-layer encoder/decoder structure. Pre…
UdS Submission for the WMT 19 Automatic Post-Editing Task
Hongfei Xu, Qiuhui Liu, Josef van Genabith
In this paper, we describe our submission to the English-German APE shared task at WMT 2019. We utilize and adapt an NMT architecture originally developed for exploiting context in…
Neutron: An Implementation of the Transformer Translation Model and its Variants
Hongfei Xu, Qiuhui Liu
The Transformer translation model is easier to parallelize and provides better performance compared to recurrent seq2seq models, which makes it popular among industry and research…