Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination

Gengze Xu, Anxin Guo, Yifei Hu +3

Understanding the relationship between generalization and hallucination requires clarifying the role of memorization. We study this relationship from an information-theoretic persp…