Comparing BFGS and OGR for Second-Order Optimization

Adrian Przybysz, Mikołaj Kołek, Franciszek Sobota +1

Estimating the Hessian matrix, especially for neural network training, is a challenging problem due to high dimensionality and cost. In this work, we compare the classical Sherman-…