5 papers
How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models
Anxin Guo, Chirag Pabbaraju, Sara Ahmadian +2
Large language models (LLMs) often exhibit sycophancy, altering their answers to align with user suggestions even on objective factual and reasoning tasks. Yet the quantitative for…
Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination
Gengze Xu, Anxin Guo, Yifei Hu +3
Understanding the relationship between generalization and hallucination requires clarifying the role of memorization. We study this relationship from an information-theoretic persp…
Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing
Anxin Guo, Jingwei Li
Large language models often hallucinate with high confidence on "random facts" that lack inferable patterns. We formalize the memorization of such facts as a membership testing pro…
Agnostic Learning of Arbitrary ReLU Activation under Gaussian Marginals
Anxin Guo, Aravindan Vijayaraghavan
We consider the problem of learning an arbitrarily-biased ReLU activation (or neuron) over Gaussian marginals with the squared loss objective. Despite the ReLU neuron being the bas…
To Store or Not to Store: a graph theoretical approach for Dataset Versioning
Anxin Guo, Jingwei Li, Pattara Sukprasert +3
In this work, we study the cost efficient data versioning problem, where the goal is to optimize the storage and reconstruction (retrieval) costs of data versions, given a graph of…