Detectable Only Where It Is Confounded: What Verified Duplication Counts Say About Membership Evidence in Language Models

Arman Nik Khah

When a language model finds a sentence unusually cheap to predict, it is tempting to conclude that the sentence was in its training data. Almost every published test of that infere…