4 papers
Sketching Low-Rank Plus Diagonal Matrices
Andres Fernandez, Felix Dangel, Philipp Hennig +1
Many relevant machine learning and scientific computing tasks involve high-dimensional linear operators accessible only via costly matrix-vector products. In this context, recent a…
Efficient Neural and Numerical Methods for High-Quality Online Speech Spectrogram Inversion via Gradient Theorem
Andres Fernandez, Juan Azcarreta, Cagdas Bilen +1
Recent work in online speech spectrogram inversion effectively combines Deep Learning with the Gradient Theorem to predict phase derivatives directly from magnitudes. Then, phases…
Connecting Parameter Magnitudes and Hessian Eigenspaces at Scale using Sketched Methods
Andres Fernandez, Frank Schneider, Maren Mahsereci +1
Recently, it has been observed that when training a deep neural net with SGD, the majority of the loss landscape's curvature quickly concentrates in a tiny *top* eigenspace of the…
Position: Curvature Matrices Should Be Democratized via Linear Operators
Felix Dangel, Runa Eschenhagen, Weronika Ormaniec +3
Structured large matrices are prevalent in machine learning. A particularly important class is curvature matrices like the Hessian, which are central to understanding the loss land…