2.4k citations · 4k across the 38 of their papers we have counts for
4 papers · 1 filter
SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint
Zhonghao Sheng, Kaitao Song, Xu Tan +4
Automatic song writing aims to compose a song (lyric and/or melody) by machine, which is an interesting topic in both academia and industry. In automatic song writing, lyric-to-mel…
Neural Machine Translation with Error Correction
Kaitao Song, Xu Tan, Jianfeng Lu
Neural machine translation (NMT) generates the next target token given as input the previous ground truth target tokens during training while the previous generated target tokens d…
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Kaitao Song, Hao Sun, Xu Tan +4
While pre-training and fine-tuning, e.g., BERT~\citep{devlin2018bert}, GPT-2~\citep{radford2019language}, have achieved great success in language understanding and generation tasks…
MPNet: Masked and Permuted Pre-training for Language Understanding
Kaitao Song, Xu Tan, Tao Qin +2
BERT adopts masked language modeling (MLM) for pre-training and is one of the most successful pre-training models. Since BERT neglects dependency among predicted tokens, XLNet intr…