21 citations · 45 across the 12 of their papers we have counts for
5 papers · 1 filter
Learning Hard Retrieval Decoder Attention for Transformers
Hongfei Xu, Qiuhui Liu, Josef van Genabith +1
The Transformer translation model is based on the multi-head attention mechanism, which can be parallelized easily. The multi-head attention network performs the scaled dot-product…
Rewiring the Transformer with Depth-Wise LSTMs
Hongfei Xu, Yang Song, Qiuhui Liu +2
Stacking non-linear layers allows deep neural networks to model complicated functions, and including residual connections in Transformer layers is beneficial for convergence and pe…
Learning Source Phrase Representations for Neural Machine Translation
Hongfei Xu, Josef van Genabith, Deyi Xiong +2
The Transformer translation model (Vaswani et al., 2017) based on a multi-head attention mechanism can be computed effectively in parallel and has significantly pushed forward the…
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change
Hongfei Xu, Josef van Genabith, Deyi Xiong +1
The choice of hyper-parameters affects the performance of neural models. While much previous research (Sutskever et al., 2013; Duchi et al., 2011; Kingma and Ba, 2015) focuses on a…
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers
Hongfei Xu, Josef van Genabith, Qiuhui Liu +1
Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches. Previous work focuses…