Showing 2026Show all
3 papers · 1 filter
cs.IT2026
Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination
Gengze Xu, Anxin Guo, Yifei Hu +3
Understanding the relationship between generalization and hallucination requires clarifying the role of memorization. We study this relationship from an information-theoretic persp…
cs.LG2026
On the Blessing of Pre-training in Weak-to-Strong Generalization
Wei Yao, Wang Zhaoyang, Gengze Xu +5
The paradigm of Weak-to-Strong Generalization (W2SG) suggests that a pre-trained strong model can surpass its weak supervisor, yet the decisive role of pre-training remains theoret…
cs.CL2026
Beyond the Black Box: A Survey on the Theory and Mechanism of Large Language Models
Zeyu Gan, Ruifeng Ren, Wei Yao +9
The rapid emergence of Large Language Models (LLMs) has precipitated a profound paradigm shift in Artificial Intelligence, delivering monumental engineering successes that increasi…