6 citations · 6 across the 7 of their papers we have counts for
4 papers · 1 filter
Fast Gauss-Newton for Multiclass Cross-Entropy
Mikalai Korbit, Mario Zanon
In multiclass softmax cross-entropy, the full generalized Gauss-Newton (GGN) curvature couples all output logits through the softmax covariance, making curvature-vector products ha…
Second-Order, First-Class: A Composable Stack for Curvature-Aware Training
Mikalai Korbit, Mario Zanon
Second-order methods promise improved stability and faster convergence, yet they remain underused due to implementation overhead, tuning brittleness, and the lack of composable API…
Incremental Gauss-Newton Descent for Machine Learning
Mikalai Korbit, Mario Zanon
Stochastic gradient updates are widely used for their efficiency and scalability, but their effective step sizes can depend strongly on feature scaling and local model sensitivity.…
Exact Gauss-Newton Optimization for Training Deep Neural Networks
Mikalai Korbit, Adeyemi D. Adeoye, Alberto Bemporad +1
We present Exact Gauss-Newton (EGN), a stochastic second-order optimization algorithm that combines the generalized Gauss-Newton (GN) Hessian approximation with low-rank linear alg…