11 citations · 36 across the 16 of their papers we have counts for
14 papers · 1 filter
The NiuTrans Machine Translation Systems for WMT21
Shuhan Zhou, Tao Zhou, Binghao Wei +15
This paper describes NiuTrans neural machine translation systems of the WMT 2021 news translation tasks. We made submissions to 9 language directions, including English$\leftrighta…
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking
Chi Hu, Chenglong Wang, Xiangnan Ma +5
This paper addresses the efficiency challenge of Neural Architecture Search (NAS) by formulating the task as a ranking problem. Previous methods require numerous training examples…
The NiuTrans System for WNGT 2020 Efficiency Task
Chi Hu, Bei Li, Ye Lin +5
This paper describes the submissions of the NiuTrans Team to the WNGT 2020 Efficiency Shared Task. We focus on the efficient implementation of deep Transformer models \cite{wang-et…
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation
Bei Li, Quan Du, Tao Zhou +4
It has been found that residual networks are an Euler discretization of solutions to Ordinary Differential Equations (ODEs). In this paper, we explore a deeper relationship between…
Non-Autoregressive Translation by Learning Target Categorical Codes
Yu Bao, Shujian Huang, Tong Xiao +3
Non-autoregressive Transformer is a promising text generation model. However, current non-autoregressive models still fall behind their autoregressive counterparts in translation q…
An Efficient Transformer Decoder with Compressed Sub-layers
Yanyang Li, Ye Lin, Tong Xiao +1
The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness. But the high computation complexity of its decoder raises t…
Learning Light-Weight Translation Models from Deep Transformer
Bei Li, Ziyang Wang, Hui Liu +4
Recently, deep models have shown tremendous improvements in neural machine translation (NMT). However, systems of this kind are computationally expensive and memory intensive. In t…
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction
Yanyang Li, Yingfeng Luo, Ye Lin +5
Unsupervised Bilingual Dictionary Induction methods based on the initialization and the self-learning have achieved great success in similar language pairs, e.g., English-Spanish.…
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation
Chen Xu, Bojie Hu, Yufan Jiang +6
Large amounts of data has made neural machine translation (NMT) a big success in recent years. But it is still a challenge if we train these models on small-scale corpora. In this…
Layer-Wise Multi-View Learning for Neural Machine Translation
Qiang Wang, Changliang Li, Yue Zhang +2
Traditional neural machine translation is limited to the topmost encoder layer's context representation and cannot directly perceive the lower encoder layers. Existing solutions us…
Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation
Qiang Wang, Tong Xiao, Jingbo Zhu
The standard neural machine translation model can only decode with the same depth configuration as training. Restricted by this feature, we have to deploy models of various sizes t…
Shallow-to-Deep Training for Neural Machine Translation
Bei Li, Ziyang Wang, Hui Liu +5
Deep encoders have been proven to be effective in improving neural machine translation (NMT) systems, but training an extremely deep encoder is time consuming. Moreover, why deep m…
Weight Distillation: Transferring the Knowledge in Neural Network Parameters
Ye Lin, Yanyang Li, Ziyang Wang +4
Knowledge distillation has been proven to be effective in model acceleration and compression. It allows a small network to learn to generalize in the same way as a large network. R…
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation
Bei Li, Hui Liu, Ziyang Wang +5
In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information in addition to the individual sentence. In this paper, we investigat…