ConvBERT: Improving BERT with Span-based Dynamic Convolution

Zihang Jiang, Weihao Yu, Daquan Zhou +3

Pre-trained language models like BERT and its variants have recently achieved impressive performance in various natural language understanding tasks. However, BERT heavily relies o…