15 citations · 55 across the 17 of their papers we have counts for
18 papers · 1 filter
A Quasi-Newton Primal-Dual Algorithm with Line Search
Shida Wang, Jalal Fadili, Peter Ochs
Quasi-Newton methods refer to a class of algorithms at the interface between first and second order methods. They aim to progress as substantially as second order methods per itera…
Continuous Newton-like Methods featuring Inertia and Variable Mass
Camille Castera, Hedy Attouch, Jalal Fadili +1
We introduce a new dynamical system, at the interface between second-order dynamics with inertia and Newton's method. This system extends the class of inertial Newton-like dynamics…
Inertial Quasi-Newton Methods for Monotone Inclusion: Efficient Resolvent Calculus and Primal-Dual Methods
Shida Wang, Jalal Fadili, Peter Ochs
We introduce an inertial quasi-Newton Forward-Backward Splitting Algorithm to solve a class of monotone inclusion problems. While the inertial step is computationally cheap, in gen…
Fixed-Point Automatic Differentiation of Forward--Backward Splitting Algorithms for Partly Smooth Functions
Sheheryar Mehmood, Peter Ochs
A large class of non-smooth practical optimization problems can be written as minimization of a sum of smooth and partly smooth functions. We examine such structured problems which…
Differentiating the Value Function by using Convex Duality
Sheheryar Mehmood, Peter Ochs
We consider the differentiation of the value function for parametric optimization problems. Such problems are ubiquitous in Machine Learning applications such as structured support…
Global Convergence of Model Function Based Bregman Proximal Minimization Algorithms
Mahesh Chandra Mukkamala, Jalal Fadili, Peter Ochs
Lipschitz continuity of the gradient mapping of a continuously differentiable function plays a crucial role in designing various optimization algorithms. However, many functions ar…
Automatic Differentiation of Some First-Order Methods in Parametric Optimization
Sheheryar Mehmood, Peter Ochs
We aim at computing the derivative of the solution to a parametric optimization problem with respect to the involved parameters. For a class broader than that of strongly convex fu…
Bregman Proximal Framework for Deep Linear Neural Networks
Mahesh Chandra Mukkamala, Felix Westerkamp, Emanuel Laude +2
A typical assumption for the analysis of first order optimization methods is the Lipschitz continuity of the gradient of the objective function. However, for many practical applica…
Bregman Proximal Mappings and Bregman-Moreau Envelopes under Relative Prox-Regularity
Emanuel Laude, Peter Ochs, Daniel Cremers
We systematically study the local single-valuedness of the Bregman proximal mapping and local smoothness of the Bregman--Moreau envelope of a nonconvex function under relative prox…
Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms
Mahesh Chandra Mukkamala, Peter Ochs
Matrix Factorization is a popular non-convex optimization problem, for which alternating minimization schemes are mostly used. They usually suffer from the major drawback that the…
Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization
Mahesh Chandra Mukkamala, Peter Ochs, Thomas Pock +1
Backtracking line-search is an old yet powerful strategy for finding a better step sizes to be used in proximal gradient algorithms. The main principle is to locally find a simple…
Model Function Based Conditional Gradient Method with Armijo-like Line Search
Yura Malitsky, Peter Ochs
The Conditional Gradient Method is generalized to a class of non-smooth non-convex optimization problems with many applications in machine learning. The proposed algorithm iterates…
On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence
Stephen Becker, Jalal Fadili, Peter Ochs
We introduce a framework for quasi-Newton forward--backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal rank- symmetric positive…
Adaptive FISTA for Non-convex Optimization
Peter Ochs, Thomas Pock
In this paper we propose an adaptively extrapolated proximal gradient method, which is based on the accelerated proximal gradient method (also known as FISTA), however we locally o…
Non-smooth Non-convex Bregman Minimization: Unification and new Algorithms
Peter Ochs, Jalal Fadili, Thomas Brox
We propose a unifying algorithm for non-smooth non-convex optimization. The algorithm approximates the objective function by a convex model function and finds an approximate (Bregm…
Local Convergence of the Heavy-ball Method and iPiano for Non-convex Optimization
Peter Ochs
A local convergence result for abstract descent methods is proved. The sequence of iterates is attracted by a local (or global) minimum, stays in its neighborhood and converges wit…
Unifying abstract inexact convergence theorems and block coordinate variable metric iPiano
Peter Ochs
An abstract convergence theorem for a class of generalized descent methods that explicitly models relative errors is proved. The convergence theorem generalizes and unifies several…
Techniques for Gradient Based Bilevel Optimization with Nonsmooth Lower Level Problems
Peter Ochs, René Ranftl, Thomas Brox +1
We propose techniques for approximating bilevel optimization problems with non-smooth lower level problems that can have a non-unique solution. To this end, we substitute the expre…