Visualizing and Understanding the Effectiveness of BERT

Yaru Hao, Li Dong, Furu Wei +1

Language model pre-training, such as BERT, has achieved remarkable results in many NLP tasks. However, it is unclear why the pre-training-then-fine-tuning paradigm can improve perf…