7 citations · 13 across the 6 of their papers we have counts for
7 papers
Contextures: Representations from Contexts
Runtian Zhai, Kai Yang, Che-Ping Tsai +3
Despite the empirical success of foundation models, we do not have a systematic characterization of the representations that these models learn. In this paper, we establish the con…
Contextures: The Mechanism of Representation Learning
Runtian Zhai
This dissertation establishes the contexture theory to mathematically characterize the mechanism of representation learning, or pretraining. Despite the remarkable empirical succes…
Spectrally Transformed Kernel Regression
Runtian Zhai, Rattana Pukdee, Roger Jin +2
Unlabeled data is a key component of modern machine learning. In general, the role of unlabeled data is to impose a form of smoothness, usually from the similarity information enco…
Responsible AI (RAI) Games and Ensembles
Yash Gupta, Runtian Zhai, Arun Suggala +1
Several recent works have studied the societal effects of AI; these include issues such as fairness, robustness, and safety. In many of these objectives, a learner seeks to minimiz…
DORO: Distributional and Outlier Robust Optimization
Runtian Zhai, Chen Dan, J. Zico Kolter +1
Many machine learning tasks involve subpopulation shift where the testing data distribution is a subpopulation of the training distribution. For such settings, a line of recent wor…
Transferred Discrepancy: Quantifying the Difference Between Representations
Yunzhen Feng, Runtian Zhai, Di He +2
Understanding what information neural networks capture is an essential problem in deep learning, and studying whether different models capture similar features is an initial step t…