8 citations · 32 across the 42 of their papers we have counts for
46 papers
From Plato to Aristotle: Toward a General Theory of Representation Hypotheses
Dai Shi, Andi Han, Xiaoyu Li +2
The Platonic Representation Hypothesis (PRH) holds that increasingly capable models converge toward a common representation. However, recent studies have shown different views on a…
Self-Confirming Superposition Traps in Reinforcement Learning
Dai Shi, Andi Han, Feng Chen +3
Reinforcement learning (RL) trains representations on data selected by the agent's policy, which then uses the resulting returns to guide its next choices. We show that this loop c…
Characterizing Language Generation in the Limit: Finite Witnesses and a Separation-Width Hierarchy
Xiaoyu Li, Andi Han, Jiaojiao Jiang +1
Language generation in the limit asks for valid unseen elements from every exhaustive positive presentation of an unknown infinite language. We characterize this task for arbitrary…
Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks
Xiaoyu Li, Zhizhou Sha, Jiaojiao Jiang +2
An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input-dependent sparsity in the on…
Feature Superposition in Neural Networks: From Theory to Practice
Dai Shi, Xiaoyu Li, Andi Han +1
Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons and motivates methods for re…
Optimistic Rates for Multiclass PAC Learning
Xiaoyu Li, Andi Han, Jiaojiao Jiang +1
Worst-case multiclass bounds do not become smaller when the best classifier is already nearly correct: what is missing is an optimistic rate, a guarantee whose fluctuation scales w…