A Review on Language Models as Knowledge Bases

Abstract

Recently, there has been a surge of interest in the NLP community on the use of pretrained Language Models (LMs) as Knowledge Bases (KBs). Researchers have shown that LMs trained on a sufficiently large (web) corpus will encode a significant amount of knowledge implicitly in its parameters. The resulting LM can be probed for different kinds of knowledge and thus acting as a KB. This has a major advantage over traditional KBs in that this method requires no human supervision. In this paper, we present a set of aspects that we deem a LM should have to fully act as a KB, and review the recent literature with respect to those aspects.

arXiv commentPreprint

Cited by in corpus (9)

• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• A Survey on Symbolic Knowledge Distillation of Large Language Models
• Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models
• Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities
• Do PLMs Know and Understand Ontological Knowledge?
• OPT-R: Exploring the Role of Explanations in Finetuning and Prompting for Reasoning Skills of Large Language Models
• Large Language Models and Multimodal Retrieval for Visual Word Sense Disambiguation
• Generative AI for Research Data Processing: Lessons Learnt From Three Use Cases
• A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia