6 papers
Understanding the Curse of Unrolling
Sheheryar Mehmood, Florian Knoll, Peter Ochs
Algorithm unrolling is ubiquitous in machine learning, particularly in hyperparameter optimization and meta-learning, where Jacobians of solution mappings are computed by different…
Symmetries in PAC-Bayesian Learning
Armin Beck, Peter Ochs
Symmetries are known to improve the empirical performance of machine learning models, yet theoretical guarantees explaining these gains remain limited. Prior work has focused mainl…
Bregman Stochastic Proximal Point Algorithm with Variance Reduction
Cheik Traoré, Peter Ochs
Stochastic algorithms, especially stochastic gradient descent (SGD), have proven to be the go-to methods in data science and machine learning. In recent years, the stochastic proxi…
Automatic Differentiation of Optimization Algorithms with Time-Varying Updates
Sheheryar Mehmood, Peter Ochs
Numerous Optimization Algorithms have a time-varying update rule thanks to, for instance, a changing step size, momentum parameter or, Hessian approximation. In this paper, we appl…
A Generalization Result for Convergence in Learning-to-Optimize
Michael Sucker, Peter Ochs
Learning-to-optimize leverages machine learning to accelerate optimization algorithms. While empirical results show tremendous improvements compared to classical optimization algor…
A Probabilistic Framework for Learnable Optimization Algorithms
Peter Ochs, Michael Sucker
We propose a statistical-learning framework for optimization algorithms. The framework is based on probability distributions over optimization trajectories induced by a distributio…