5 papers
TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation
Ziheng Chen, Jiali Cheng, Zezhong Fan +4
Generative recommendation formulates next-item prediction as autoregressive generation over semantic ID (SID) sequences derived from users' historical interactions, making modern r…
Improved Scaling Laws via Weak-to-Strong Generalization in Random Feature Ridge Regression
Diyuan Wu, Lehan Chen, Theodor Misiakiewicz +1
It is increasingly common in machine learning to use learned models to label data and then employ such data to train more capable models. The phenomenon of weak-to-strong generaliz…
Rethinking Crystal Symmetry Prediction: A Decoupled Perspective
Liheng Yu, Zhe Zhao, Xucong Wang +2
Efficiently and accurately determining the symmetry is a crucial step in the structural analysis of crystalline materials. Existing methods usually mindlessly apply deep learning m…
Attention with Trained Embeddings Provably Selects Important Tokens
Diyuan Wu, Aleksandr Shevchenko, Samet Oymak +1
Token embeddings play a crucial role in language modeling but, despite this practical relevance, their theoretical understanding remains limited. Our paper addresses the gap by cha…
Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime
Diyuan Wu, Marco Mondelli
Neural Collapse is a phenomenon where the last-layer representations of a well-trained neural network converge to a highly structured geometry. In this paper, we focus on its first…