7 papers
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Zhenyu Liao, Michael W. Mahoney
Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions…
Dual-Attention Convolution Experts for Sparse Tensor Completion
Yanlei Liu, Zhenyu Liao
Tensor factorization (TF) has been widely adopted for high-dimensional sparse data completion tasks. Despite significant progress, neural TF methods often struggle to capture compl…
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko +1
We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs. By heuristically extending the Convex Gaussian Min-Max Theorem (CGMT) to…
Debiasing Random Oblique Projections for Subsampled OLS and Fast CUR in High Dimensions
Chengmei Niu, Sachin Garg, MichaÅ DereziÅski +1
Random sampling is a fundamental tool in modern machine learning and numerical linear algebra for reducing the computational cost of large-scale matrix problems. Existing analyses,…
Fundamental Bias in Inverting Random Sampling Matrices with Application to Sub-sampled Newton
Chengmei Niu, Zhenyu Liao, Zenan Ling +1
A substantial body of work in machine learning (ML) and randomized numerical linear algebra (RandNLA) has exploited various sorts of random sketching methodologies, including rando…
On the Interpolation Error of Nonlinear Attention versus Linear Regression
Zhenyu Liao, Jiaqing Liu, TianQi Hou +2
Attention has become the core building block of modern machine learning (ML) by efficiently capturing the long-range dependencies among input tokens. Its inherently parallelizable…