Distilling Task-Specific Knowledge from BERT into Simple Neural Networks

Raphael Tang, Yao Lu, Linqing Liu +3

In the natural language processing literature, neural networks are becoming increasingly deeper and complex. The recent poster child of this trend is the deep language representati…