4 papers
Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization
Chris Kolb, Christian L. Müller, Bernd Bischl +1
We present a framework for smooth optimization of explicitly regularized objectives for (structured) sparsity. These non-smooth and possibly non-convex problems typically rely on s…
On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors
Julius Kobialka, Emanuel Sommer, Chris Kolb +3
Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space…
Differentiable Sparsity via -Gating: Simple and Versatile Structured Penalization
Chris Kolb, Laetitia Frost, Bernd Bischl +1
Structured sparsity regularization offers a principled way to compact neural networks, but its non-differentiability breaks compatibility with conventional stochastic gradient desc…
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries
Chris Kolb, Tobias Weber, Bernd Bischl +1
Sparse regularization techniques are well-established in machine learning, yet their application in neural networks remains challenging due to the non-differentiability of penaltie…