Measuring and Improving Consistency in Pretrained Language Models
arXiv:2102.01017
Abstract
Consistency of a model -- that is, the invariance of its behavior under meaning-preserving alternations in its input -- is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create ParaRel, a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for 38 relations. Using ParaRel, we show that the consistency of all PLMs we experiment with is poor -- though with high variance between relations. Our analysis of the representational spaces of PLMs suggests that they have a poor structure and are currently not suitable for representing knowledge robustly. Finally, we propose a method for improving model consistency and experimentally demonstrate its effectiveness.
arXiv commentAccepted to the TACL journal, pre-MIT Press publication version
References in corpus (11)
- Scikit-learn: Machine Learning in Python
- Language Models are Few-Shot Learners
- VQA: Visual Question Answering
- Assessing BERT's Syntactic Abilities
- What do Neural Machine Translation Models Learn about Morphology?
- e-SNLI: Natural Language Inference with Natural Language Explanations
- How Context Affects Language Models' Factual Predictions
- Commonsense Knowledge Mining from Pretrained Models
- Measuring Inconsistency in Probabilistic Knowledge Bases
- Neural Databases
- Enriching a Model's Notion of Belief using a Persistent Memory
Cited by in corpus (12)
- Unsolved Problems in ML Safety
- Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings
- Few-Shot Text Generation with Pattern-Exploiting Training
- Accurate, yet inconsistent? Consistency Analysis on Language Understanding Models
- Enriching a Model's Notion of Belief using a Persistent Memory
- Relational World Knowledge Representation in Contextual Language Models: A Review
- BERTnesia: Investigating the capture and forgetting of knowledge in BERT
- P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts
- Continuous Entailment Patterns for Lexical Inference in Context
- Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems
- On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation
- Sorting through the noise: Testing robustness of information processing in pre-trained language models