4 papers · 1 filter
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko +1
We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs. By heuristically extending the Convex Gaussian Min-Max Theorem (CGMT) to…
On the Interpolation Error of Nonlinear Attention versus Linear Regression
Zhenyu Liao, Jiaqing Liu, TianQi Hou +2
Attention has become the core building block of modern machine learning (ML) by efficiently capturing the long-range dependencies among input tokens. Its inherently parallelizable…
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Zhenyu Liao, Michael W. Mahoney
Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions…
Analysis and Approximate Inference of Large Random Kronecker Graphs
Zhenyu Liao, Yuanqian Xia, Chengmei Niu +1
Random graph models are playing an increasingly important role in various fields ranging from social networks, telecommunication systems, to physiologic and biological networks. Wi…