Adaptive Regularization for Sparsity Control in Bregman-Based Optimizers

Ahmad Aloradi, Tim Roith, Emanuël A. P. Habets +1

Sparse training reduces the memory and computational costs of deep neural networks. However, sparse optimization methods, e.g., those adding an ${\ell }_1$ penalty, often control spar…