4 papers
On the Recoverability of Private Information Unlearning in Large Language Models
Shicheng Hu, Runzhi Tian, Ziqiao Wang +1
Large language models (LLMs) can memorize sensitive information, raising serious privacy concerns. Machine unlearning offers a potential solution to remove such information, but it…
On the Cost and Benefit of Chain of Thought: A Learning-Theoretic Perspective
Yue Zhang, Zhiyi Dong, Tommaso Cesari +1
We develop a learning-theoretic framework for understanding Chain of Thought (CoT). We model CoT as the interaction between an answer map and a chain rule that generates intermedia…
Parallel Recursive LSTM
Tristan Gaudreault, Yongyi Mao
Transformers have become the dominant architecture for sequence modeling by using self-attention to enable expressive and highly parallel processing. However, the resulting quadrat…
On robust overfitting: adversarial training induced distribution matters
Runzhi Tian, Yongyi Mao
Adversarial training may be regarded as standard training with a modified loss function. But its generalization error appears much larger than standard training under standard loss…