82 citations · 167 across the 13 of their papers we have counts for
13 papers
Contextures: Representations from Contexts
Runtian Zhai, Kai Yang, Che-Ping Tsai +3
Despite the empirical success of foundation models, we do not have a systematic characterization of the representations that these models learn. In this paper, we establish the con…
Contextures: The Mechanism of Representation Learning
Runtian Zhai
This dissertation establishes the contexture theory to mathematically characterize the mechanism of representation learning, or pretraining. Despite the remarkable empirical succes…
Spectrally Transformed Kernel Regression
Runtian Zhai, Rattana Pukdee, Roger Jin +2
Unlabeled data is a key component of modern machine learning. In general, the role of unlabeled data is to impose a form of smoothness, usually from the similarity information enco…
Responsible AI (RAI) Games and Ensembles
Yash Gupta, Runtian Zhai, Arun Suggala +1
Several recent works have studied the societal effects of AI; these include issues such as fairness, robustness, and safety. In many of these objectives, a learner seeks to minimiz…
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression
Runtian Zhai, Bingbin Liu, Andrej Risteski +2
Data augmentation is critical to the empirical success of modern self-supervised representation learning, such as contrastive learning and masked language modeling. However, a theo…
Characterizing Out-of-Distribution Error via Optimal Transport
Yuzhe Lu, Yilong Qin, Runtian Zhai +7
Out-of-distribution (OOD) data poses serious challenges in deployed machine learning models, so methods of predicting a model's performance on OOD data without labels are important…
Predicting Out-of-Distribution Error with Confidence Optimal Transport
Yuzhe Lu, Zhenlin Wang, Runtian Zhai +3
Out-of-distribution (OOD) data poses serious challenges in deployed machine learning models as even subtle changes could incur significant performance drops. Being able to estimate…
Understanding Why Generalized Reweighting Does Not Improve Over ERM
Runtian Zhai, Chen Dan, Zico Kolter +1
Empirical risk minimization (ERM) is known in practice to be non-robust to distributional shift where the training and the test distributions are different. A suite of approaches,…
Boosted CVaR Classification
Runtian Zhai, Chen Dan, Arun Sai Suggala +2
Many modern machine learning tasks require models with high tail performance, i.e. high performance over the worst-off samples in the dataset. This problem has been widely studied…
DORO: Distributional and Outlier Robust Optimization
Runtian Zhai, Chen Dan, J. Zico Kolter +1
Many machine learning tasks involve subpopulation shift where the testing data distribution is a subpopulation of the training distribution. For such settings, a line of recent wor…
Transferred Discrepancy: Quantifying the Difference Between Representations
Yunzhen Feng, Runtian Zhai, Di He +2
Understanding what information neural networks capture is an essential problem in deep learning, and studying whether different models capture similar features is an initial step t…
MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Runtian Zhai, Chen Dan, Di He +5
Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly. In this paper, we propose the MACER algorithm, which le…
Adversarially Robust Generalization Just Requires More Unlabeled Data
Runtian Zhai, Tianle Cai, Di He +4
Neural network robustness has recently been highlighted by the existence of adversarial examples. Many previous works show that the learned networks do not perform well on perturbe…