BERTnesia: Investigating the capture and forgetting of knowledge in BERT

Abstract

Probing complex language models has recently revealed several insights into linguistic and semantic patterns found in the learned representations. In this paper, we probe BERT specifically to understand and measure the relational knowledge it captures. We utilize knowledge base completion tasks to probe every layer of pre-trained as well as fine-tuned BERT (ranking, question answering, NER). Our findings show that knowledge is not just contained in BERT's final layers. Intermediate layers contribute a significant amount (17-60%) to the total knowledge found. Probing intermediate layers also reveals how different types of knowledge emerge at varying rates. When BERT is fine-tuned, relational knowledge is forgotten but the extent of forgetting is impacted by the fine-tuning objective but not the size of the dataset. We found that ranking models forget the least and retain more knowledge in their final layer. We release our code on github to repeat the experiments.

arXiv commentBBNLP 2020, Edit: Small correction in the setup section, no changes to data or findings presented in the paper

References in corpus (21)

• Decoupled Weight Decay Regularization
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Axiomatic Attribution for Deep Networks
• ERNIE: Enhanced Representation through Knowledge Integration
• REALM: Retrieval-Augmented Language Model Pre-Training
• Generating Long Sequences with Sparse Transformers
• Passage Re-ranking with BERT
• Assessing BERT's Syntactic Abilities
• Know What You Don't Know: Unanswerable Questions for SQuAD
• What do you learn from context? Probing for sentence structure in contextualized word representations
• K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
• Pre-training Tasks for Embedding-based Large-scale Retrieval
• What Do Compressed Deep Neural Networks Forget?
• Explain and Predict, and then Predict Again
• Posthoc Interpretability of Learning to Rank Models using Secondary Training Data
• Measuring and Improving Consistency in Pretrained Language Models
• DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
• PMI-Masking: Principled masking of correlated spans
• Do Neural Language Representations Learn Physical Commonsense?
• Enriching a Model's Notion of Belief using a Persistent Memory
• Not All Memories are Created Equal: Learning to Forget by Expiring