2 citations · 4 across the 8 of their papers we have counts for
8 papers
Mitigating Forgetting in Low Rank Adaptation
Joanna Sliwa, Frank Schneider, Philipp Hennig +1
Parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), enable fast specialization of large pre-trained models to different downstream applications. However, t…
Sketching Low-Rank Plus Diagonal Matrices
Andres Fernandez, Felix Dangel, Philipp Hennig +1
Many relevant machine learning and scientific computing tasks involve high-dimensional linear operators accessible only via costly matrix-vector products. In this context, recent a…
Connecting Parameter Magnitudes and Hessian Eigenspaces at Scale using Sketched Methods
Andres Fernandez, Frank Schneider, Maren Mahsereci +1
Recently, it has been observed that when training a deep neural net with SGD, the majority of the loss landscape's curvature quickly concentrates in a tiny *top* eigenspace of the…
Accelerating Neural Network Training: An Analysis of the AlgoPerf Competition
Priya Kasimbeg, Frank Schneider, Runa Eschenhagen +11
The goal of the AlgoPerf: Training Algorithms competition is to evaluate practical speed-ups in neural network training achieved solely by improving the underlying training algorit…
Low-Rank Filtering and Smoothing for Sequential Deep Learning
Joanna Sliwa, Frank Schneider, Nathanael Bosch +2
Learning multiple tasks sequentially requires neural networks to balance retaining knowledge, yet being flexible enough to adapt to new tasks. Regularizing network parameters is a…
Uncertainty-Guided Likelihood Tree Search
Julia Grosse, Ruotian Wu, Ahmad Rashid +4
Tree search is a fundamental tool for planning, as many sequential decision-making problems can be framed as searching over tree-structured spaces. We propose an uncertainty-guided…
Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
Runa Eschenhagen, Alexander Immer, Richard E. Turner +2
The core components of many modern neural network architectures, such as transformers, convolutional, or graph neural networks, can be expressed as linear layers with $\textit{weig…
Accelerating Non-Conjugate Gaussian Processes By Trading Off Computation For Uncertainty
Lukas Tatzel, Jonathan Wenger, Frank Schneider +1
Non-conjugate Gaussian processes (NCGPs) define a flexible probabilistic framework to model categorical, ordinal and continuous data, and are widely used in practice. However, exac…