Language Models As or For Knowledge Bases
arXiv:2110.04888
Abstract
Pre-trained language models (LMs) have recently gained attention for their potential as an alternative to (or proxy for) explicit knowledge bases (KBs). In this position paper, we examine this hypothesis, identify strengths and limitations of both LMs and KBs, and discuss the complementary nature of the two paradigms. In particular, we offer qualitative arguments that latent LMs are not suitable as a substitute for explicit KBs, but could play a major role for augmenting and curating KBs.
References in corpus (7)
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- REALM: Retrieval-Augmented Language Model Pre-Training
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
- Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases
- Language Models are Open Knowledge Graphs
- Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
- Enriching a Model's Notion of Belief using a Persistent Memory