5 papers
Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization
Chris Kolb, Christian L. Müller, Bernd Bischl +1
We present a framework for smooth optimization of explicitly regularized objectives for (structured) sparsity. These non-smooth and possibly non-convex problems typically rely on s…
Differentiable Sparsity via -Gating: Simple and Versatile Structured Penalization
Chris Kolb, Laetitia Frost, Bernd Bischl +1
Structured sparsity regularization offers a principled way to compact neural networks, but its non-differentiability breaks compatibility with conventional stochastic gradient desc…
Semi-Implicit Variational Inference via Kernelized Path Gradient Descent
Tobias Pielok, Bernd Bischl, David Rügamer
Semi-implicit variational inference (SIVI) is a powerful framework for approximating complex posterior distributions, but training with the Kullback-Leibler (KL) divergence can be…
Revisiting Unbiased Implicit Variational Inference
Tobias Pielok, Bernd Bischl, David Rügamer
Recent years have witnessed growing interest in semi-implicit variational inference (SIVI) methods due to their ability to rapidly generate samples from complex distributions. Howe…
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries
Chris Kolb, Tobias Weber, Bernd Bischl +1
Sparse regularization techniques are well-established in machine learning, yet their application in neural networks remains challenging due to the non-differentiability of penaltie…