335 citations · 395 across the 6 of their papers we have counts for
Showing 2019 · cs.CLShow all
2 papers · 2 filters
cs.CL2019
MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
Linqing Liu, Huan Wang, Jimmy Lin +2
Pretrained language models have led to significant performance gains in many NLP tasks. However, the intensive computing resources to train such models remain an issue. Knowledge d…
cs.CL2019★ 335 cited
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
Raphael Tang, Yao Lu, Linqing Liu +3
In the natural language processing literature, neural networks are becoming increasingly deeper and complex. The recent poster child of this trend is the deep language representati…