36 citations · 155 across the 14 of their papers we have counts for
6 papers · 1 filter
Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods
Thomas O'Leary-Roseberry, Raghu Bollapragada
We consider minimizing finite-sum and expectation objective functions via Hessian-averaging based subsampled Newton methods. These methods allow for gradient inexactness and have f…
Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators
Dingcheng Luo, Thomas O'Leary-Roseberry, Peng Chen +1
We propose a novel machine learning framework for solving optimization problems governed by large-scale partial differential equations (PDEs) with high-dimensional random parameter…
Ill-Posedness and Optimization Geometry for Nonlinear Neural Network Training
Thomas O'Leary-Roseberry, Omar Ghattas
In this work we analyze the role nonlinear activation functions play at stationary points of dense neural network training problems. We consider a generic least squares loss functi…
Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization
Thomas O'Leary-Roseberry, Nick Alger, Omar Ghattas
In modern deep learning, highly subsampled stochastic approximation (SA) methods are preferred to sample average approximation (SAA) methods because of large data sets as well as g…
Inexact Newton Methods for Stochastic Nonconvex Optimization with Applications to Neural Network Training
Thomas O'Leary-Roseberry, Nick Alger, Omar Ghattas
We study stochastic inexact Newton methods and consider their application in nonconvex settings. Building on the work of [R. Bollapragada, R. H. Byrd, and J. Nocedal, IMA Journal o…
Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions
Peng Chen, Keyi Wu, Joshua Chen +2
We propose a fast and scalable variational method for Bayesian inference in high-dimensional parameter space, which we call projected Stein variational Newton (pSVN) method. We exp…