10 citations · 10 across the 16 of their papers we have counts for
7 papers · 1 filter
Layers Matter: Why Continual Learning Regularization Should Be Layer-Adaptive
Brian B. Moser, Ahmed Anwar, Tobias Christian Nauen +5
Continual learning regularizers like EWC fight forgetting by penalizing changes from previous-task parameters with per-parameter importance, typically diagonal Fisher values. Per-p…
The Gentle Collapse: Distributional Metrics for Continual Learning
Ahmed Anwar, Andreas Wagner, Federico Raue +2
Accuracy degradation is the standard metric for Catastrophic Forgetting (CF), however, it records only whether forgetting occurred or not. It saturates at the extremes and collapse…
Hyperspherical Forward-Forward with Prototypical Representations
Shalini Sarode, Brian Moser, Joachim Folz +4
The Forward-Forward (FF) algorithm presents a compelling, bio-inspired alternative to backpropagation. However, while efficient in training, it has a computationally prohibitive in…
PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors
Brian B. Moser, Shalini Sarode, Federico Raue +6
Dataset distillation (DD) promises compact yet faithful synthetic data, but existing approaches often inherit the inductive bias of a single teacher model. As dataset size increase…
SubZeroCore: A Submodular Approach with Zero Training for Coreset Selection
Brian B. Moser, Tobias C. Nauen, Arundhati S. Shanbhag +4
The goal of coreset selection is to identify representative subsets of datasets for efficient model training. Yet, existing approaches paradoxically require expensive training-base…
HyperCore: Coreset Selection under Noise via Hypersphere Models
Brian B. Moser, Arundhati S. Shanbhag, Tobias C. Nauen +4
The goal of coreset selection methods is to identify representative subsets of datasets for efficient model training. Yet, existing methods often ignore the possibility of annotati…
TaylorShift: Shifting the Complexity of Self-Attention from Squared to Linear (and Back) using Taylor-Softmax
Tobias Christian Nauen, Sebastian Palacio, Andreas Dengel
The quadratic complexity of the attention mechanism represents one of the biggest hurdles for processing long sequences using Transformers. Current methods, relying on sparse repre…