1.2k citations · 9.1k across the 205 of their papers we have counts for
27 papers · 1 filter
A Study of Multilingual Neural Machine Translation
Xu Tan, Yichong Leng, Jiale Chen +3
Multilingual neural machine translation (NMT) has recently been investigated from different aspects (e.g., pivot translation, zero-shot translation, fine-tuning, or training from s…
Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
Junliang Guo, Xu Tan, Linli Xu +3
Non-autoregressive translation (NAT) models remove the dependence on previous target tokens and generate all target tokens in parallel, resulting in significant inference speedup b…
Microsoft Research Asia's Systems for WMT19
Yingce Xia, Xu Tan, Fei Tian +11
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks. We won the first place for 8 of the 11 directions and the second place fo…
Distributional Reward Decomposition for Reinforcement Learning
Zichuan Lin, Li Zhao, Derek Yang +3
Many reinforcement learning (RL) tasks have specific properties that can be leveraged to modify existing RL algorithms to adapt to those tasks and further improve performance, and…
Fully Parameterized Quantile Function for Distributional Reinforcement Learning
Derek Yang, Li Zhao, Zichuan Lin +3
Distributional Reinforcement Learning (RL) differs from traditional RL in that, rather than the expectation of total returns, it estimates distributions and has achieved state-of-t…
Hint-Based Training for Non-Autoregressive Machine Translation
Zhuohan Li, Zi Lin, Di He +4
Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer…
Self-paced Ensemble for Highly Imbalanced Massive Data Classification
Zhining Liu, Wei Cao, Zhifeng Gao +4
Many real-world applications reveal difficulties in learning classifiers from imbalanced data. The rising big data era has been witnessing more classification tasks with large-scal…
LightMC: A Dynamic and Efficient Multiclass Decomposition Algorithm
Ziyu Liu, Guolin Ke, Jiang Bian +1
Multiclass decomposition splits a multiclass classification problem into a series of independent binary learners and recomposes them by combining their outputs to reconstruct the m…
Multilingual Neural Machine Translation with Language Clustering
Xu Tan, Jiale Chen, Di He +3
Multilingual neural machine translation (NMT), which translates multiple languages using a single model, is of great practical importance due to its advantages in simplifying the t…
Representation Degeneration Problem in Training Natural Language Generation Models
Jun Gao, Di He, Xu Tan +3
We study an interesting problem in training neural network-based models for natural language generation tasks, which we call the \emph{representation degeneration problem}. We obse…
Depth Growing for Neural Machine Translation
Lijun Wu, Yiren Wang, Yingce Xia +5
While very deep neural networks have shown effectiveness for computer vision and text classification applications, how to increase the network depth of neural machine translation (…
Light Multi-segment Activation for Model Compression
Zhenhui Xu, Guolin Ke, Jia Zhang +2
Model compression has become necessary when applying neural networks (NN) into many real application tasks that can accept slightly-reduced model accuracy with strict tolerance to…
Unsupervised Pivot Translation for Distant Languages
Yichong Leng, Xu Tan, Tao Qin +2
Unsupervised neural machine translation (NMT) has attracted a lot of attention recently. While state-of-the-art methods for unsupervised translation usually perform well between si…
MASS: Masked Sequence to Sequence Pre-training for Language Generation
Kaitao Song, Xu Tan, Tao Qin +2
Pre-training and fine-tuning, e.g., BERT, have achieved great success in language understanding by transferring knowledge from rich-resource pre-training task to the low/zero-resou…
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
Yiping Lu, Zhuohan Li, Di He +5
The Transformer architecture is widely used in natural language processing. Despite its success, the design principle of the Transformer remains elusive. In this paper, we provide…
Learning Efficient and Effective Exploration Policies with Counterfactual Meta Policy
Ruihan Yang, Qiwei Ye, Tie-Yan Liu
A fundamental issue in reinforcement learning algorithms is the balance between exploration of the environment and exploitation of information already obtained by the agent. Especi…
Soft Contextual Data Augmentation for Neural Machine Translation
Jinhua Zhu, Fei Gao, Lijun Wu +5
While data augmentation is an important trick to boost the accuracy of deep learning methods in computer vision tasks, its study in natural language tasks is still very limited. In…
Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data
Shicong Cen, Huishuai Zhang, Yuejie Chi +2
Stochastic variance reduced methods have gained a lot of interest recently for empirical risk minimization due to its appealing run time complexity. When the data size is large and…
Beyond Exponentially Discounted Sum: Automatic Learning of Return Function
Yufei Wang, Qiwei Ye, Tie-Yan Liu
In reinforcement learning, Return, which is the weighted accumulated future rewards, and Value, which is the expected return, serve as the objective that guides the learning of the…
Almost Unsupervised Text to Speech and Automatic Speech Recognition
Yi Ren, Xu Tan, Tao Qin +3
Text to speech (TTS) and automatic speech recognition (ASR) are two dual tasks in speech processing and both achieve impressive performance thanks to the recent advance in deep lea…
Adaptive Regret of Convex and Smooth Functions
Lijun Zhang, Tie-Yan Liu, Zhi-Hua Zhou
We investigate online convex optimization in changing environments, and choose the adaptive regret as the performance measure. The goal is to achieve a small regret over every inte…
Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
Hao Sun, Xu Tan, Jun-Wei Gan +4
Grapheme-to-phoneme (G2P) conversion is an important task in automatic speech recognition and text-to-speech systems. Recently, G2P conversion is viewed as a sequence to sequence t…
Positively Scale-Invariant Flatness of ReLU Neural Networks
Mingyang Yi, Qi Meng, Wei Chen +2
It was empirically confirmed by Keskar et al.\cite{SharpMinima} that flatter minima generalize better. However, for the popular ReLU network, sharp minimum can also generalize well…
A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Network
Xihan Li, Jia Zhang, Jiang Bian +2
Resource balancing within complex transportation networks is one of the most important problems in real logistics domain. Traditional solutions on these problems leverage combinato…