Aligning Large Language Models with Human: A Survey

Abstract

Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect (hallucinated) information. Hence, aligning LLMs with human expectations has become an active area of interest within the research community. This survey presents a comprehensive overview of these alignment technologies, including the following aspects. (1) Data collection: the methods for effectively collecting high-quality instructions for LLM alignment, including the use of NLP benchmarks, human annotations, and leveraging strong LLMs. (2) Training methodologies: a detailed review of the prevailing training methods employed for LLM alignment. Our exploration encompasses Supervised Fine-tuning, both Online and Offline human preference training, along with parameter-efficient training mechanisms. (3) Model Evaluation: the methods for evaluating the effectiveness of these human-aligned LLMs, presenting a multifaceted approach towards their assessment. In conclusion, we collate and distill our findings, shedding light on several promising future research avenues in the field. This survey, therefore, serves as a valuable resource for anyone invested in understanding and advancing the alignment of LLMs to better suit human-oriented tasks and expectations. An associated GitHub link collecting the latest papers is available at https://github.com/GaryYufei/AlignLLMHumanSurvey.

arXiv commentwork in progress

Cited by in corpus (9)

• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• How understanding large language models can inform the use of ChatGPT in physics education
• A Survey on Symbolic Knowledge Distillation of Large Language Models
• A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges
• From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models
• The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies
• Assessing the Potential of Generative Agents in Crowdsourced Fact-Checking
• The Fair Game: Auditing & Debiasing AI Algorithms Over Time
• Data2Concept2Text: An Explainable Multilingual Framework for Data Analysis Narration