Learning Dynamic BERT via Trainable Gate Variables and a Bi-modal Regularizer

Seohyeong Jeong, Nojun Kwak

The BERT model has shown significant success on various natural language processing tasks. However, due to the heavy model size and high computational cost, the model suffers from…