How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training

Yixin Ou, Yunzhi Yao, Ningyu Zhang +5

Despite exceptional capabilities in knowledge-intensive tasks, Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge, particularly ho…