Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective

Chenye Ke, Zirui Liu, Qi Liu +4

Detecting pretraining data in large language models is challenging because high likelihood can reflect either training exposure or strong generalization. In the joint space of pred…