3 papers
math.OC2025
Greedy Newton: Newton's Method with Exact Line Search
Betty Shea, Mark Schmidt
{A defining characteristic of Newton's method is local superlinear convergence within a neighbourhood of a strict local minimum. However, outside this neighborhood Newton's method…
cs.LG2024
Don't Be So Positive: Negative Step Sizes in Second-Order Methods
Betty Shea, Mark Schmidt
The value of second-order methods lies in the use of curvature information. Yet, this information is costly to extract and once obtained, valuable negative curvature information is…
cs.LG2024
Why Line Search when you can Plane Search? SO-Friendly Neural Networks allow Per-Iteration Optimization of Learning and Momentum Rates for Every Layer
Betty Shea, Mark Schmidt
We introduce the class of SO-friendly neural networks, which include several models used in practice including networks with 2 layers of hidden weights where the number of inputs i…