1.9k citations · 2.7k across the 13 of their papers we have counts for
6 papers · 1 filter
Reward Collapse in Aligning Large Language Models
Ziang Song, Tianle Cai, Jason D. Lee +1
The extraordinary capabilities of large language models (LLMs) such as ChatGPT and GPT-4 are in part unleashed by aligning them with reward models that are trained on human prefere…
A Law of Data Separation in Deep Learning
Hangfeng He, Weijie J. Su
While deep learning has enabled significant advances in many areas of science, its black-box nature hinders architecture design for future artificial intelligence applications and…
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data
Zhun Deng, Jiayao Zhang, Linjun Zhang +4
Algorithmic fairness plays an important role in machine learning and imposing fairness constraints during learning is a common approach. However, many datasets are imbalanced in ce…
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations
Jiayao Zhang, Hua Wang, Weijie J. Su
Understanding the training dynamics of deep learning models is perhaps a necessary step toward demystifying the effectiveness of these models. In particular, how do data from diffe…
You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism
Weijie J. Su
I consider a setting where reviewers offer very noisy scores for several items for the selection of high-quality ones (e.g., peer review of large conference proceedings), whereas t…
An Unconstrained Layer-Peeled Perspective on Neural Collapse
Wenlong Ji, Yiping Lu, Yiliang Zhang +2
Neural collapse is a highly symmetric geometric pattern of neural networks that emerges during the terminal phase of training, with profound implications on the generalization perf…