372 citations · 862 across the 46 of their papers we have counts for
18 papers · 1 filter
Vocabulary Learning via Optimal Transport for Neural Machine Translation
Jingjing Xu, Hao Zhou, Chun Gan +2
The choice of token vocabulary affects the performance of machine translation. This paper aims to figure out what is a good vocabulary and whether one can find the optimal vocabula…
CPM: A Large-scale Generative Chinese Pre-trained Language Model
Zhengyan Zhang, Xu Han, Hao Zhou +22
Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB training data, drew a lot o…
On the Sentence Embeddings from Pre-trained Language Models
Bohan Li, Hao Zhou, Junxian He +3
Pre-trained contextual representations like BERT have achieved great success in natural language processing. However, the sentence embeddings from the pre-trained language models w…
NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions
Zhiyu Chen, Honglei Liu, Hu Xu +3
Existing conversational systems are mostly agent-centric, which assumes the user utterances would closely follow the system ontology (for NLU or dialogue state tracking). However,…
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information
Zehui Lin, Xiao Pan, Mingxuan Wang +4
We investigate the following question for machine translation (MT): can we develop a single universal MT model to serve as the common seed and obtain derivative and improved models…
Consecutive Decoding for Speech-to-text Translation
Qianqian Dong, Mingxuan Wang, Hao Zhou +3
Speech-to-text translation (ST), which directly translates the source language speech to the target language text, has attracted intensive attention recently. However, the combinat…
"Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text Translation
Qianqian Dong, Rong Ye, Mingxuan Wang +4
An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel cor…
Glancing Transformer for Non-Autoregressive Neural Machine Translation
Lihua Qian, Hao Zhou, Yu Bao +5
Recent work on non-autoregressive neural machine translation (NAT) aims at improving the efficiency by parallel decoding without sacrificing the quality. However, existing NAT meth…
Generating Fluent Adversarial Examples for Natural Languages
Huangzhao Zhang, Hao Zhou, Ning Miao +1
Efficiently building an adversarial attacker for natural language processing (NLP) tasks is a real challenge. Firstly, as the sentence space is discrete, it is difficult to make sm…
Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods
Ning Miao, Yuxuan Song, Hao Zhou +1
It has been a common approach to pre-train a language model on a large corpus and fine-tune it on task-specific data. In practice, we observe that fine-tuning a pre-trained model o…
Xiaomingbot: A Multilingual Robot News Reporter
Runxin Xu, Jun Cao, Mingxuan Wang +10
This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translat…
Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimation
Yuxuan Song, Ning Miao, Hao Zhou +3
Auto-regressive sequence generative models trained by Maximum Likelihood Estimation suffer the exposure bias problem in practical finite sample scenarios. The crux is that the numb…
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization
Xunpeng Huang, Runxin Xu, Hao Zhou +3
Due to its simplicity and outstanding ability to generalize, stochastic gradient descent (SGD) is still the most widely used optimization method despite its slow convergence. Meanw…
Adaptive Gradient Methods Can Be Provably Faster than SGD after Finite Epochs
Xunpeng Huang, Hao Zhou, Runxin Xu +2
Adaptive gradient methods have attracted much attention of machine learning communities due to the high efficiency. However their acceleration effect in practice, especially in neu…
KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation
Hao Zhou, Chujie Zheng, Kaili Huang +2
The research of knowledge-driven conversational systems is largely limited due to the lack of dialog data which consist of multi-turn conversations on multiple topics and with know…
Infomax Neural Joint Source-Channel Coding via Adversarial Bit Flip
Yuxuan Song, Minkai Xu, Lantao Yu +3
Although Shannon theory states that it is asymptotically optimal to separate the source and channel coding as two independent processes, in many practical communication scenarios t…
Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space
Dongyu Ru, Jiangtao Feng, Lin Qiu +5
Active learning for sentence understanding aims at discovering informative unlabeled data for annotation and therefore reducing the demand for labeled data. We argue that the typic…
Variational Template Machine for Data-to-Text Generation
Rong Ye, Wenxian Shi, Hao Zhou +2
How to generate descriptions from structured data organized in tables? Existing approaches using neural encoder-decoder models often suffer from lacking diversity. We claim that an…