Large Knowledge Model: Perspectives and Challenges

Abstract

Humankind's understanding of the world is fundamentally linked to our perception and cognition, with \emph{human languages} serving as one of the major carriers of \emph{world knowledge}. In this vein, \emph{Large Language Models} (LLMs) like ChatGPT epitomize the pre-training of extensive, sequence-based world knowledge into neural networks, facilitating the processing and manipulation of this knowledge in a parametric space. This article explores large models through the lens of "knowledge". We initially investigate the role of symbolic knowledge such as Knowledge Graphs (KGs) in enhancing LLMs, covering aspects like knowledge-augmented language model, structure-inducing pre-training, knowledgeable prompts, structured CoT, knowledge editing, semantic tools for LLM and knowledgeable AI agents. Subsequently, we examine how LLMs can boost traditional symbolic knowledge bases, encompassing aspects like using LLM as KG builder and controller, structured knowledge pretraining, and LLM-enhanced symbolic reasoning. Considering the intricate nature of human knowledge, we advocate for the creation of \emph{Large Knowledge Models} (LKM), specifically engineered to manage diversified spectrum of knowledge structures. This promising undertaking would entail several key challenges, such as disentangling knowledge base from language models, cognitive alignment with human knowledge, integration of perception and cognition, and building large commonsense models for interacting with physical world, among others. We finally propose a five-"A" principle to distinguish the concept of LKM.

Data Intelligence, Published: Jun 18, 2024

References in corpus (56)

• Training language models to follow instructions with human feedback
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• A Survey of Large Language Models
• Scaling Laws for Neural Language Models
• Emergent Abilities of Large Language Models
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Toolformer: Language Models Can Teach Themselves to Use Tools
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• Large Language Models Are Human-Level Prompt Engineers
• Measuring Mathematical Problem Solving With the MATH Dataset
• HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
• The Rise and Potential of Large Language Model Based Agents: A Survey
• Automatic Chain of Thought Prompting in Large Language Models
• Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
• ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT
• Augmented Language Models: a Survey
• Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
• K-BERT: Enabling Language Representation with Knowledge Graph
• KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion
• Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
• Exploring the Feasibility of ChatGPT for Event Extraction
• Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
• Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
• OntoProtein: Protein Pretraining With Gene Ontology Embedding
• Integrating Graph Contextualized Knowledge into Pre-trained Language Models
• LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities
• Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey
• ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations
• Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning
• Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
• A Comprehensive Study of Knowledge Editing for Large Language Models
• Agents: An Open-source Framework for Autonomous Language Agents
• Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey
• ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualities
• Making Large Language Models Perform Better in Knowledge Graph Completion
• GraphGPT: Graph Instruction Tuning for Large Language Models
• Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective
• $k$NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference
• MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models
• EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models
• T-RAG: Lessons from the LLM Trenches
• Making Language Models Better Tool Learners with Execution Feedback
• InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
• InstructIE: A Bilingual Instruction-based Information Extraction Dataset
• Neural-Symbolic Entangled Framework for Complex Query Answering
• reStructured Pre-training
• Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering
• Knowledge-aware Zero-Shot Learning: Survey and Perspective
• Detoxifying Large Language Models via Knowledge Editing
• KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents
• NLQxform: A Language Model-based Question to SPARQL Transformer
• When Do Program-of-Thoughts Work for Reasoning?
• Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models
• Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models

Cited by in corpus (2)

• SPARK: Adaptive Low-Rank Knowledge Graph Modeling in Hybrid Geometric Spaces for Recommendation
• Generative AI and the future of scientometrics: current topics and future questions