44 citations · 153 across the 39 of their papers we have counts for
42 papers
Equilibrium Learning in Unbounded Stochastic Games with Gaussian Transitions
Jonathan Anthony Mavroforas, Fred Roosta
Learning equilibria from data is difficult when several decision makers interact indefinitely and the system state has no prescribed bounds. We examine discounted general-sum stoch…
Why Heavy-Tailed Weight Spectra Predict Model Quality
Joseph Wilson, Chris van der Heide, Liam Hodgkinson +3
A solid understanding of the predictive success of deep neural networks (DNNs) remains elusive. For example, while many data-dependent metrics exist that seek to predict model qual…
Primal-Dual Inexact Newton-MR for Nonconvex Optimization with Equality Constraints
Oscar Smee, Fred Roosta
Optimization problems with nonlinear equality constraints arise throughout science, engineering, and increasingly in machine learning. Prominent methods for solving such problems i…
Is the Last Layer Sufficient for Uncertainty Quantification?
Joseph Wilson, Chris van der Heide, Liam Hodgkinson +1
Epistemic uncertainty quantification (UQ) for deep neural networks (DNNs) is a requirement for safe adoption of AI in mission-critical settings. Several leading methods for UQ line…
The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
Eslam Zaher, Maciej Trzaskowski, Quan Nguyen +1
Sparse autoencoders (SAEs) operationalise the linear representation hypothesis: they reconstruct model activations as sparse linear combinations of interpretable dictionary atoms,…
Counterfactual Explanations on Robust Perceptual Geodesics
Eslam Zaher, Maciej Trzaskowski, Quan Nguyen +1
Latent-space optimization methods for counterfactual explanations - framed as minimal semantic perturbations that change model predictions - inherit the ambiguity of Wachter et al.…
Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving
Hossein Askari, Yadan Luo, Hongfu Sun +1
Recent advances in inverse problem solving have increasingly adopted flow priors over diffusion models due to their ability to construct straight probability paths from noise to da…
Faithful-Newton Framework: Bridging Inner and Outer Solvers for Enhanced Optimization
Alexander Lim, Fred Roosta
Newton-type methods enjoy fast local convergence and strong empirical performance, but achieving global guarantees comparable to first-order methods remains challenging. Even for s…
Importance Sampling for Nonlinear Models
Prakash Palanivelu Rajmohan, Fred Roosta
While norm-based and leverage-score-based methods have been extensively studied for identifying "important" data points in linear models, analogous tools for nonlinear models remai…
Determinant Estimation under Memory Constraints and Neural Scaling Laws
Siavash Ameli, Chris van der Heide, Liam Hodgkinson +2
Calculating or accurately estimating log-determinants of large positive definite matrices is of fundamental importance in many machine learning tasks. While its cubic computational…
First-ish Order Methods: Hessian-aware Scalings of Gradient Descent
Oscar Smee, Fred Roosta, Stephen J. Wright
Gradient descent is the primary workhorse for optimizing large-scale problems in machine learning. However, its performance is highly sensitive to the choice of the learning rate.…
Uncertainty Quantification with the Empirical Neural Tangent Kernel
Joseph Wilson, Chris van der Heide, Liam Hodgkinson +1
While neural networks have demonstrated impressive performance across various tasks, accurately quantifying uncertainty in their predictions is essential to ensure their trustworth…
Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints
Oscar Smee, Fred Roosta
We consider solving large scale nonconvex optimisation problems with nonnegativity constraints. Such problems arise frequently in machine learning, such as nonnegative least-square…
Manifold Integrated Gradients: Riemannian Geometry for Feature Attribution
Eslam Zaher, Maciej Trzaskowski, Quan Nguyen +1
In this paper, we dive into the reliability concerns of Integrated Gradients (IG), a prevalent feature attribution method for black-box deep learning models. We particularly addres…
Bi-level Guided Diffusion Models for Zero-Shot Medical Imaging Inverse Problems
Hossein Askari, Fred Roosta, Hongfu Sun
In the realm of medical imaging, inverse problems aim to infer high-quality images from incomplete, noisy measurements, with the objective of minimizing expenses and risks to patie…
Conjugate Direction Methods Under Inconsistent Systems
Alexander Lim, Yang Liu, Fred Roosta
Since the development of the conjugate gradient (CG) method in 1952 by Hestenes and Stiefel, CG, has become an indispensable tool in computational mathematics for solving positive…
SALSA: Sequential Approximate Leverage-Score Algorithm with Application in Analyzing Big Time Series Data
Ali Eshragh, Luke Yerbury, Asef Nazari +2
We develop a new efficient sequential approximate leverage score algorithm, SALSA, using methods from randomized numerical linear algebra (RandNLA) for large matrices. We demonstra…
Non-Uniform Smoothness for Gradient Descent
Albert S. Berahas, Lindon Roberts, Fred Roosta
The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient. This generally requires an expensive hyperparameter tuning pro…
A PAC-Bayesian Perspective on the Interpolating Information Criterion
Liam Hodgkinson, Chris van der Heide, Robert Salomone +2
Deep learning is renowned for its theory-practice gap, whereby principled theory typically fails to provide much beneficial guidance for implementation in practice. This has been h…
Obtaining Pseudo-inverse Solutions With MINRES
Yang Liu, Andre Milzarek, Fred Roosta
The celebrated minimum residual method (MINRES), proposed in the seminal paper of Paige and Saunders, has seen great success and widespread use in solving Hermitian (and complex-sy…
Complexity Guarantees for Nonconvex Newton-MR Under Inexact Hessian Information
Alexander Lim, Fred Roosta
We consider an extension of the Newton-MR algorithm for nonconvex unconstrained optimization to the settings where Hessian information is approximated. Under a particular noise mod…
The Interpolating Information Criterion for Overparameterized Models
Liam Hodgkinson, Chris van der Heide, Robert Salomone +2
The problem of model selection is considered for the setting of interpolating estimators, where the number of model parameters exceeds the size of the dataset. Classical informatio…
Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes
Liam Hodgkinson, Chris van der Heide, Fred Roosta +1
Despite their importance for assessing reliability of predictions, uncertainty quantification (UQ) measures for machine learning models have only recently begun to be rigorously ch…
A Newton-MR algorithm with complexity guarantees for nonconvex smooth unconstrained optimization
Yang Liu, Fred Roosta
In this paper, we consider variants of Newton-MR algorithm for solving unconstrained, smooth, but non-convex optimization problems. Unlike the overwhelming majority of Newton-type…