169 citations · 176 across the 3 of their papers we have counts for
3 papers
cs.CL2020★ 5 cited
LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding
Hao Fu, Shaojun Zhou, Qihong Yang +4
The pre-training models such as BERT have achieved great results in various natural language processing problems. However, a large number of parameters need significant amounts of…
cs.CL2020★ 2 cited
Improving Text Generation with Student-Forcing Optimal Transport
Guoyin Wang, Chunyuan Li, Jianqiao Li +10
Neural language models are often trained with maximum likelihood estimation (MLE), where the next word is generated conditioned on the ground-truth word tokens. During testing, how…
cs.LG2019★ 169 cited
Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
Hao Fu, Chunyuan Li, Xiaodong Liu +3
Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks. The VAE objective consists of two terms, (i) rec…