159 citations · 910 across the 32 of their papers we have counts for
8 papers · 1 filter
MixKD: Towards Efficient Distillation of Large-scale Language Models
Kevin J Liang, Weituo Hao, Dinghan Shen +4
Large-scale language models have recently demonstrated impressive empirical performance. Nevertheless, the improved results are attained at the price of bigger models, more power c…
A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation
Dinghan Shen, Mingzhi Zheng, Yelong Shen +2
Adversarial training has been shown effective at endowing the learned representations with stronger generalization ability. However, it typically requires expensive computation to…
CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
Yanru Qu, Dinghan Shen, Yelong Shen +3
Data augmentation has been demonstrated as an effective strategy for improving model generalization and data efficiency. However, due to the discrete nature of natural language, de…
Improving Self-supervised Pre-training via a Fully-Explored Masked Language Model
Mingzhi Zheng, Dinghan Shen, Yelong Shen +2
Masked Language Model (MLM) framework has been widely adopted for self-supervised language pre-training. In this paper, we argue that randomly sampled masks in MLM would lead to un…
Improving Text Generation with Student-Forcing Optimal Transport
Guoyin Wang, Chunyuan Li, Jianqiao Li +10
Neural language models are often trained with maximum likelihood estimation (MLE), where the next word is generated conditioned on the ground-truth word tokens. During testing, how…
Generative Semantic Hashing Enhanced via Boltzmann Machines
Lin Zheng, Qinliang Su, Dinghan Shen +1
Generative semantic hashing is a promising technique for large-scale information retrieval thanks to its fast retrieval speed and small memory footprint. For the tractability of tr…