Improving BERT with Hybrid Pooling Network and Drop Mask

Qian Chen, Wen Wang, Qinglin Zhang +3

Transformer-based pre-trained language models, such as BERT, achieve great success in various natural language understanding tasks. Prior research found that BERT captures a rich h…