5 papers
Muon is Not That Special: Random or Inverted Spectra Work Just as Well
Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6
The recent empirical success of the Muon optimizer has renewed interest in non-Euclidean optimization, typically justified by similarities with second-order methods, and linear min…
Closed-Form Last Layer Optimization
Alexandre Galashov, Nathaël Da Costa, Liyuan Xu +2
Neural networks are typically optimized with variants of stochastic gradient descent. Under a squared loss, however, the optimal solution to the linear last layer weights is known…
Invariant kernels on Riemannian symmetric spaces: a harmonic-analytic approach
Nathael Da Costa, Cyrus Mostajeran, Juan-Pablo Ortega +1
This work aims to prove that the classical Gaussian kernel, when defined on a non-Euclidean symmetric space, is never positive-definite for any choice of parameter. To achieve this…
Geometric Learning with Positively Decomposable Kernels
Nathael Da Costa, Cyrus Mostajeran, Juan-Pablo Ortega +1
Kernel methods are powerful tools in machine learning. Classical kernel methods are based on positive-definite kernels, which map data spaces into reproducing kernel Hilbert spaces…
The Gaussian kernel on the circle and spaces that admit isometric embeddings of the circle
Nathaël Da Costa, Cyrus Mostajeran, Juan-Pablo Ortega
On Euclidean spaces, the Gaussian kernel is one of the most widely used kernels in applications. It has also been used on non-Euclidean spaces, where it is known that there may be…