8 papers · 1 filter
Eigen-Spike Emergence and Quadratic Equivalents for Conjugate Kernels on Nonlinearly Separable Data
Collin Cranston, Zhichao Wang, Todd Kemp +1
Recent work in random matrix theory (RMT) has developed the notion of deterministic equivalents: typically linear surrogate models that approximate the spectral behavior of large n…
Free Decompression with Algebraic Spectral Curves
Siavash Ameli, Chris van der Heide, Liam Hodgkinson +1
Tools from random matrix theory have become central to deep learning theory, using spectral information to provide mechanisms for modeling generalization, robustness, scaling, and…
Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning
Mara Daniels, Liam Hodgkinson, Michael Mahoney
Physics-informed machine learning (PIML) integrates prior physical information, often in the form of differential equation constraints, into the process of fitting machine learning…
Spectral Estimation with Free Decompression
Siavash Ameli, Chris van der Heide, Liam Hodgkinson +1
Computing eigenvalues of very large matrices is a critical task in many machine learning applications, including the evaluation of log-determinants, the trace of matrix functions,…
Models of Heavy-Tailed Mechanistic Universality
Liam Hodgkinson, Zhichao Wang, Michael W. Mahoney
Recent theoretical and empirical successes in deep learning, including the celebrated neural scaling laws, are punctuated by the observation that many objects of interest tend to e…
Determinant Estimation under Memory Constraints and Neural Scaling Laws
Siavash Ameli, Chris van der Heide, Liam Hodgkinson +2
Calculating or accurately estimating log-determinants of large positive definite matrices is of fundamental importance in many machine learning tasks. While its cubic computational…