335 citations · 504 across the 9 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.CL2019★ 19 cited
MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
Linqing Liu, Huan Wang, Jimmy Lin +2
Pretrained language models have led to significant performance gains in many NLP tasks. However, the intensive computing resources to train such models remain an issue. Knowledge d…
cs.CL2019★ 335 cited
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
Raphael Tang, Yao Lu, Linqing Liu +3
In the natural language processing literature, neural networks are becoming increasingly deeper and complex. The recent poster child of this trend is the deep language representati…