Natural Language Processing with Commonsense Knowledge: A Survey

Abstract

Commonsense knowledge is essential for advancing natural language processing (NLP) by enabling models to engage in human-like reasoning, which requires a deeper understanding of context and often involves making inferences based on implicit external knowledge. This paper explores the integration of commonsense knowledge into various NLP tasks. We begin by reviewing prominent commonsense knowledge bases and then discuss the benchmarks used to evaluate the commonsense reasoning capabilities of NLP models, particularly language models. Furthermore, we highlight key methodologies for incorporating commonsense knowledge and their applications across different NLP tasks. The paper also examines the challenges and emerging trends in enhancing NLP systems with commonsense reasoning. All literature referenced in this survey can be accessed via our GitHub repository: https://github.com/yuboxie/awesome-commonsense.

20 pages, 3 figures, 1 table

References in corpus (27)

• Enhanced LSTM for Natural Language Inference
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
• The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
• A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models
• Aligning AI With Shared Human Values
• Mass-Editing Memory in a Transformer
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
• A Dataset for Answering Time-Sensitive Questions
• Universal Multimodal Representation for Language Understanding
• Understanding Social Reasoning in Language Models with Language Models
• Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?
• Evaluating the External and Parametric Knowledge Fusion of Large Language Models
• Commonsense Reasoning for Conversational AI: A Survey of the State of the Art
• COKE: A Cognitive Knowledge Graph for Machine Theory of Mind
• TRAM: Benchmarking Temporal Reasoning for Large Language Models
• Knowledge Editing for Large Language Models: A Survey
• Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering
• Commonsense-Aware Prompting for Controllable Empathetic Dialogue Generation
• Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarios
• Multi-turn Response Selection with Commonsense-enhanced Language Models
• Leveraging Explicit Reasoning for Inference Integration in Commonsense-Augmented Dialogue Models
• Open-ended Commonsense Reasoning with Unrestricted Answer Scope