14 papers
Tensor Envelopes in the Nonconvex Setting
Alireza Kabgani, Masoud Ahookhosh
We introduce a tensor envelope (TEE) for composite objectives , built from a th-order Taylor model of the smooth term regularized by a th power with . To cons…
Beyond Conventional Federated Learning via High-Order Regularization
Alireza Kabgani, Masoud Ahookhosh
Federated clients that perform several local optimization steps can return parameter displacements with widely different magnitudes. The quadratic regularization of FedProx grows l…
Difference-of-Convex Optimization via Inexact Smoothing Descent Methods: Difference of High-Order Moreau Envelopes
Alireza Kabgani, Moslem Zamani, Masoud Ahookhosh
This paper studies difference-of-convex (DC) optimization problems through smoothing descent techniques. In particular, we introduce the difference of high-order Moreau envelopes (…
Relative Weak Convexity and Projected Subgradient Methods: Analysis and Convergence
Morteza Rahimi, Masoud Ahookhosh
We introduce the class of relatively weakly convex functions, which extends the classical notion of weak convexity by measuring nonconvexity relative to a distance-generating funct…
Speeding Up Nonsmooth Bayesian MCMC Sampling via Inexact Proximal Unadjusted Langevin Algorithm
Susan Ghaderi, Alireza Kabgani, Yves Moreau +1
We study sampling from posterior distributions with nonsmooth composite potentials, a setting in which proximal-based Langevin methods are theoretically appealing but in practice l…
Robust Learning Meets Quasar-Convex Optimization: Inexact High-Order Proximal-Point Methods
Alireza Kabgani, Felipe Lara, Masoud Ahookhosh
Robust learning aims to maintain model performance under noise, corruption, and distributional shifts, which are prevalent in modern machine learning applications. This work shows…
Quasar-Convex Optimization: Fundamental Properties and High-Order Proximal-Point Methods
Masoud Ahookhosh, Jose M. M. de Brito, Alireza Kabgani +2
We study the optimization of (strongly) quasar-convex functions, a class that arises naturally in many machine learning and data science applications due to its favorable propertie…
(Adaptive) Scaled gradient methods beyond locally Holder smoothness: Lyapunov analysis, convergence rate and complexity
Susan Ghaderi, Morteza Rahimi, Yves Moreau +1
This paper addresses the unconstrained minimization of smooth convex functions whose gradients are locally Holder continuous. Building on these results, we analyze the Scaled Gradi…
On fundamental properties of high-order forward-backward envelope
Alireza Kabgani, Masoud Ahookhosh
This paper studies the fundamental properties of the high-order forward-backward splitting mapping (HiFBS) and its associated high-order forward-backward envelope (HiFBE) through t…
Minimizing Smooth Kurdyka-Łojasiewicz Functions via Generalized Descent Methods: Convergence Rate and Complexity
Masoud Ahookhosh, Susan Ghaderi, Alireza Kabgani +1
This paper introduces a generalized descent algorithm (DEAL) for minimizing smooth nonconvex functions. If the objective function is nonsmooth, a smoothing technique (e.g., forward…
Inexact Levenberg-Marquardt methods under Hölder metric subregularity
Bas Symoens, Morteza Rahimi, Masoud Ahookhosh
This paper investigates two inexact Levenberg-Marquardt (LM) methods for solving systems of nonlinear equations. Both approaches compute approximate search directions by solving th…
Second-order methods for provably escaping strict saddle points in composite nonconvex and nonsmooth optimization
Alexander Bodard, Masoud Ahookhosh, Panagiotis Patrinos
This study introduces two second-order methods designed to provably avoid saddle points in composite nonconvex optimization problems: (i) a nonsmooth trust-region method and (ii) a…
Projected subgradient methods for paraconvex optimization: Application to robust low-rank matrix recovery
Morteza Rahimi, Susan Ghaderi, Yves Moreau +1
This paper is devoted to the class of paraconvex functions and presents some of its fundamental properties, characterization, and examples that can be used for their recognition an…
Matrix Completion via Nonsmooth Regularization of Fully Connected Neural Networks
Sajad Faramarzi, Farzan Haddadi, Sajjad Amini +2
Conventional matrix completion methods approximate the missing values by assuming the matrix to be low-rank, which leads to a linear approximation of missing values. It has been sh…