BERTnesia: Investigating the capture and forgetting of knowledge in BERT

Jonas Wallat, Jaspreet Singh, Avishek Anand

Probing complex language models has recently revealed several insights into linguistic and semantic patterns found in the learned representations. In this paper, we probe BERT spec…