Learning to Remove: Towards Isotropic Pre-trained BERT Embedding

Yuxin Liang, Rui Cao, Jie Zheng +2

Pre-trained language models such as BERT have become a more common choice of natural language processing (NLP) tasks. Research in word representation shows that isotropic embedding…