4 papers
Reducing Memorisation in Generative Models via Riemannian Bayesian Inference
Johanna Marie Gegenfurtner, Albert Kjøller Jacobsen, Naima Elosegui Borras +2
Modern generative models can produce realistic samples, however, balancing memorisation and generalisation remains an open problem. We approach this challenge from a Bayesian persp…
Staying on the Manifold: Geometry-Aware Noise Injection
Albert Kjøller Jacobsen, Johanna Marie Gegenfurtner, Georgios Arvanitidis
It has been shown that perturbing the input during training implicitly regularises the gradient of the learnt function, leading to smoother models and enhancing generalisation. How…
Monge SAM: Robust Reparameterization-Invariant Sharpness-Aware Minimization Based on Loss Geometry
Albert Kjøller Jacobsen, Georgios Arvanitidis
Recent studies on deep neural networks show that flat minima of the loss landscape correlate with improved generalization. Sharpness-aware minimization (SAM) efficiently finds flat…
How Redundant Is the Transformer Stack in Speech Representation Models?
Teresa Dorszewski, Albert Kjøller Jacobsen, Lenka Tětková +1
Self-supervised speech representation models, particularly those leveraging transformer architectures, have demonstrated remarkable performance across various tasks such as speech…