259 citations · 2.9k across the 123 of their papers we have counts for
131 papers
The Interpolating Information Criterion for Overparameterized Models
Liam Hodgkinson, Chris van der Heide, Robert Salomone +2
The problem of model selection is considered for the setting of interpolating estimators, where the number of model parameters exceeds the size of the dataset. Classical informatio…
A Heavy-Tailed Algebra for Probabilistic Programming
Feynman Liang, Liam Hodgkinson, Michael W. Mahoney
Despite the successes of probabilistic models based on passing noise through neural networks, recent work has identified that such methods often fail to capture tail behavior accur…
Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior
Shashank Subramanian, Peter Harrington, Kurt Keutzer +4
Pre-trained machine learning (ML) models have shown great performance for a wide range of applications, in particular in natural language processing (NLP) and computer vision (CV).…
Randomized Numerical Linear Algebra : A Perspective on the Field With an Eye to Software
Riley Murray, James Demmel, Michael W. Mahoney +10
Randomized numerical linear algebra - RandNLA, for short - concerns the use of randomization as a resource to develop improved algorithms for large-scale linear algebra computation…
Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems
Yuchen Fang, Sen Na, Michael W. Mahoney +1
We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic eq…
GACT: Activation Compressed Training for Generic Network Architectures
Xiaoxuan Liu, Lianmin Zheng, Dequan Wang +9
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.…