4 papers · 1 filter
On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models
Yi Zhang, Zhenyu Liao, Jingfeng Wu +1
Despite the widespread adoption of deterministic samplers in diffusion models (DMs), their potential limitations remain largely unexplored. In this paper, we identify collapse erro…
On the Interpolation Error of Nonlinear Attention versus Linear Regression
Zhenyu Liao, Jiaqing Liu, TianQi Hou +2
Attention has become the core building block of modern machine learning (ML) by efficiently capturing the long-range dependencies among input tokens. Its inherently parallelizable…
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Zhenyu Liao, Michael W. Mahoney
Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions…
Fundamental Bias in Inverting Random Sampling Matrices with Application to Sub-sampled Newton
Chengmei Niu, Zhenyu Liao, Zenan Ling +1
A substantial body of work in machine learning (ML) and randomized numerical linear algebra (RandNLA) has exploited various sorts of random sketching methodologies, including rando…