Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
Don't Be So Positive: Negative Step Sizes in Second-Order Methods
Betty Shea, Mark Schmidt
The value of second-order methods lies in the use of curvature information. Yet, this information is costly to extract and once obtained, valuable negative curvature information is…
cs.LG2024
Why Line Search when you can Plane Search? SO-Friendly Neural Networks allow Per-Iteration Optimization of Learning and Momentum Rates for Every Layer
Betty Shea, Mark Schmidt
We introduce the class of SO-friendly neural networks, which include several models used in practice including networks with 2 layers of hidden weights where the number of inputs i…