Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation

Bowen Wu, Huan Zhang, Mengyuan Li +4

Recently, BERT has become an essential ingredient of various NLP deep models due to its effectiveness and universal-usability. However, the online deployment of BERT is often block…