3 papers
cs.LG2026
Optimizer choice matters for the emergence of Neural Collapse
Jim Zhao, Tin Sum Cheng, Wojciech Masarczyk +1
Neural Collapse (NC) refers to the emergence of highly symmetric geometric structures in the representations of deep neural networks during the terminal phase of training. Despite…
cs.LG2025
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks
Jim Zhao, Sidak Pal Singh, Aurelien Lucchi
The Gauss-Newton (GN) matrix plays an important role in machine learning, most evident in its use as a preconditioning matrix for a wide family of popular adaptive methods to speed…
cs.LG2025
Cubic regularized subspace Newton for non-convex optimization
Jim Zhao, Aurelien Lucchi, Nikita Doikov
This paper addresses the optimization problem of minimizing non-convex continuous functions, which is relevant in the context of high-dimensional machine learning applications char…