8 papers
Dual-Attention Convolution Experts for Sparse Tensor Completion
Yanlei Liu, Zhenyu Liao
Tensor factorization (TF) has been widely adopted for high-dimensional sparse data completion tasks. Despite significant progress, neural TF methods often struggle to capture compl…
Debiasing Random Oblique Projections for Subsampled OLS and Fast CUR in High Dimensions
Chengmei Niu, Sachin Garg, Michał Dereziński +1
Random sampling is a fundamental tool in modern machine learning and numerical linear algebra for reducing the computational cost of large-scale matrix problems. Existing analyses,…
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko +1
We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs. By heuristically extending the Convex Gaussian Min-Max Theorem (CGMT) to…
On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models
Yi Zhang, Zhenyu Liao, Jingfeng Wu +1
Despite the widespread adoption of deterministic samplers in diffusion models (DMs), their potential limitations remain largely unexplored. In this paper, we identify collapse erro…
On the Interpolation Error of Nonlinear Attention versus Linear Regression
Zhenyu Liao, Jiaqing Liu, TianQi Hou +2
Attention has become the core building block of modern machine learning (ML) by efficiently capturing the long-range dependencies among input tokens. Its inherently parallelizable…
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Zhenyu Liao, Michael W. Mahoney
Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions…
Fundamental Bias in Inverting Random Sampling Matrices with Application to Sub-sampled Newton
Chengmei Niu, Zhenyu Liao, Zenan Ling +1
A substantial body of work in machine learning (ML) and randomized numerical linear algebra (RandNLA) has exploited various sorts of random sketching methodologies, including rando…
Analysis and Approximate Inference of Large Random Kronecker Graphs
Zhenyu Liao, Yuanqian Xia, Chengmei Niu +1
Random graph models are playing an increasingly important role in various fields ranging from social networks, telecommunication systems, to physiologic and biological networks. Wi…