A Smoothing Accelerated Proximal Gradient Method with Nonmonotone Backtracking Strategy for Convex Nonsmooth Composite Multiobjective Optimization

Abstract

This paper studies convex nonsmooth composite multiobjective optimization problems, whose objectives combine a convex term admitting a smooth approximation with a proper, closed, and convex term. A natural way is to smooth the nonsmooth term and apply proximal gradient method, but reducing the smoothing parameter shrinks step sizes, and solving successive smooth approximations to approach the original solution is computationally expensive. To address this issues, we propose a smoothing accelerated proximal gradient method with nonmonotone backtracking strategy. Each outer iteration starts from a reduced base smoothness estimate, which is increased only if the criterion fails, while the momentum update is linked to the estimate through an exact identity used in the convergence analysis. Under a compact level-set assumption and bounds of parameters, we use merit function to establish rates of $\mathcal{O}(k^{-\sigma })$, $\mathcal{O}((\log k)/k)$, and $\mathcal{O}(k^{\sigma -2})$ for different smoothing parameters. Then quasi-Fejér monotonicity yields convergence of the full sequence to a weak Pareto solution. Numerical experiments show that the method can reduce computational cost while preserving high-quality Pareto fronts and examine how the smoothing parameter affects performance.