Extracting and Understanding the Superficial Knowledge in Alignment

Runjin Chen, Gabriel Jacob Perin, Xuxi Chen +5

Alignment of large language models (LLMs) with human values and preferences, often achieved through fine-tuning based on human feedback, is essential for ensuring safe and responsi…