537 citations · 902 across the 19 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
eess.AS2020★ 4 cited
Speech enhancement aided end-to-end multi-task learning for voice activity detection
Xu Tan, Xiao-Lei Zhang
Robust voice activity detection (VAD) is a challenging task in low signal-to-noise (SNR) environments. Recent studies show that speech enhancement is helpful to VAD, but the perfor…
cs.CL2020★ 9 cited
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Kaitao Song, Hao Sun, Xu Tan +4
While pre-training and fine-tuning, e.g., BERT~\citep{devlin2018bert}, GPT-2~\citep{radford2019language}, have achieved great success in language understanding and generation tasks…
cs.CL2020★ 537 cited
MPNet: Masked and Permuted Pre-training for Language Understanding
Kaitao Song, Xu Tan, Tao Qin +2
BERT adopts masked language modeling (MLM) for pre-training and is one of the most successful pre-training models. Since BERT neglects dependency among predicted tokens, XLNet intr…