10 papers
Towards a theory of inference-time alignment with unknown rewards
Steve Hanneke, Hongao Wang, Mingyue Xu
Generative model alignment has received broad interest, and significant progress has been made in supervised fine-tuning and inference-time computation. Yet, alignment has remained…
Optimal Learning Under Tsybakov Noise
Steve Hanneke, Hongao Wang, Mingyue Xu
Probably Approximately Correct (PAC) learning [Val84] is a fundamental learning model that has been extensively investigated. In this model, $\mathcal{H} \subseteq \{0,1\}^{\mathca…
Attention-based representations for multi-task computation
Daniel Hsu, Mingyue Xu
Multi-head attention layers produce vector representations that support multiple downstream tasks. We establish bounds on the number of heads required in two simple and concrete mu…
When More Data Doesn't Help: Limits of Adaptation in Multitask Learning
Steve Hanneke, Mingyue Xu
Multitask learning and related frameworks have achieved tremendous success in modern applications. In multitask learning problem, we are given a set of heterogeneous datasets colle…
To Grok Grokking: Provable Grokking in Ridge Regression
Mingyue Xu, Gal Vardi, Itay Safran
We study grokking, the onset of generalization long after overfitting, in a classical ridge regression setting. We prove end-to-end grokking results for learning over-parameterized…
Universal rates of ERM for agnostic learning
Steve Hanneke, Mingyue Xu
The universal learning framework has been developed to obtain guarantees on the learning rates that hold for any fixed distribution, which can be much faster than the ones uniforml…
Rethinking Gradient-based Adversarial Attacks on Point Cloud Classification
Jun Chen, Xinke Li, Mingyue Xu +2
Gradient-based adversarial attacks are widely used to evaluate the robustness of 3D point cloud classifiers, yet they often rely on uniform update rules that neglect point-wise het…
Universal Rates of Empirical Risk Minimization
Steve Hanneke, Mingyue Xu
The well-known empirical risk minimization (ERM) principle is the basis of many widely used machine learning algorithms, and plays an essential role in the classical PAC theory. A…
Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products
Gan Yuan, Mingyue Xu, Samory Kpotufe +1
We consider the problem of sufficient dimension reduction (SDR) for multi-index models. The estimators of the central mean subspace in prior works either have slow (non-parametric)…
Distributed Semi-Supervised Sparse Statistical Inference
Jiyuan Tu, Weidong Liu, Xiaojun Mao +1
The debiased estimator is a crucial tool in statistical inference for high-dimensional model parameters. However, constructing such an estimator involves estimating the high-dimens…