1.2k citations · 9.1k across the 205 of their papers we have counts for
20 papers · 1 filter
Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
Junliang Guo, Xu Tan, Di He +3
Non-autoregressive translation (NAT) models, which remove the dependence on previous target tokens from the inputs of the decoder, achieve significantly inference speedup but at th…
Modeling Local Dependence in Natural Language with Multi-channel Recurrent Neural Networks
Chang Xu, Weiran Huang, Hongwei Wang +2
Recurrent Neural Networks (RNNs) have been widely used in processing natural language tasks and achieve huge success. Traditional RNNs usually treat each token in a sentence unifor…
Learning to Teach with Dynamic Loss Functions
Lijun Wu, Fei Tian, Yingce Xia +4
Teaching is critical to human society: it is with teaching that prospective students are educated and human civilization can be inherited and advanced. A good teacher not only prov…
Finite Sample Analysis of the GTD Policy Evaluation Algorithms in Markov Setting
Yue Wang, Wei Chen, Yuting Liu +2
In reinforcement learning (RL) , one of the key components is policy evaluation, which aims to estimate the value function (i.e., expected long-term accumulated reward) of a policy…
Target Transfer Q-Learning and Its Convergence Analysis
Yue Wang, Qi Meng, Wei Cheng +3
Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve…
Capacity Control of ReLU Neural Networks by Basis-path Norm
Shuxin Zheng, Qi Meng, Huishuai Zhang +3
Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property o…
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
Lijun Wu, Xu Tan, Di He +4
Neural machine translation usually adopts autoregressive models and suffers from exposure bias as well as the consequent error propagation problem. Many previous works have discuss…
FRAGE: Frequency-Agnostic Word Representation
Chengyue Gong, Di He, Xu Tan +3
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks. Although it is widely a…
A Study of Reinforcement Learning for Neural Machine Translation
Lijun Wu, Fei Tian, Tao Qin +2
Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation (NMT) system. However, due to its in…
Neural Architecture Optimization
Renqian Luo, Fei Tian, Tao Qin +2
Automatic neural architecture design has shown its potential in discovering powerful neural network architectures. Existing methods, no matter based on reinforcement learning or ev…
Double Path Networks for Sequence to Sequence Learning
Kaitao Song, Xu Tan, Di He +3
Encoder-decoder based Sequence to Sequence learning (S2S) has made remarkable progress in recent years. Different network architectures have been used in the encoder/decoder. Among…
Dense Information Flow for Neural Machine Translation
Yanyao Shen, Xu Tan, Di He +2
Recently, neural machine translation has achieved remarkable progress by introducing well-designed deep neural networks into its encoder-decoder framework. From the optimization pe…
Towards Binary-Valued Gates for Robust LSTM Training
Zhuohan Li, Di He, Fei Tian +4
Long Short-Term Memory (LSTM) is one of the most widely used recurrent structures in sequence modeling. It aims to use gates to control information flow (e.g., whether to skip some…
Learning to Teach
Yang Fan, Fei Tian, Tao Qin +2
Teaching plays a very important role in our society, by spreading human knowledge and educating our next generations. A good teacher will select appropriate teaching materials, imp…
Differential Equations for Modeling Asynchronous Algorithms
Li He, Qi Meng, Wei Chen +2
Asynchronous stochastic gradient descent (ASGD) is a popular parallel optimization algorithm in machine learning. Most theoretical analysis on ASGD take a discrete view and prove u…
Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling
Chenyan Xiong, Zhengzhong Liu, Jamie Callan +1
This paper presents a Kernel Entity Salience Model (KESM) that improves text understanding and retrieval by better estimating entity salience (importance) in documents. KESM repres…
Conditional Image-to-Image Translation
Jianxin Lin, Yingce Xia, Tao Qin +2
Image-to-image translation tasks have been widely investigated with Generative Adversarial Networks (GANs) and dual learning. However, existing models lack the ability to control t…
Achieving Human Parity on Automatic Chinese to English News Translation
Hany Hassan, Anthony Aue, Chang Chen +21
Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate acr…
Train Feedfoward Neural Network with Layer-wise Adaptive Rate via Approximating Back-matching Propagation
Huishuai Zhang, Wei Chen, Tie-Yan Liu
Stochastic gradient descent (SGD) has achieved great success in training deep neural network, where the gradient is computed through back-propagation. However, the back-propagated…
-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space
Qi Meng, Shuxin Zheng, Huishuai Zhang +3
It is well known that neural networks with rectified linear units (ReLU) activation functions are positively scale-invariant. Conventional algorithms like stochastic gradient desce…