Combining fast inertial dynamics for convex optimization with Tikhonov regularization

Abstract

In a Hilbert space setting $\mathcal{H}$, we study the convergence properties as $t\to +\infty$ of the trajectories of the second-order differential equation \begin{equation*} \mbox{(AVD)}_{α, ε} \quad \quad \ddot{x}(t) + \fracα{t} \dot{x}(t) + \nabla Φ(x(t)) + ε(t) x(t) =0, \end{equation*} where $\nabla \Phi$ is the gradient of a convex continuously differentiable function $\Phi :\mathcal{H}\to \mathbb{R}$, $\alpha$ is a positive parameter, and $\epsilon (t)x(t)$ is a Tikhonov regularization term, with ${\lim }_{t\to \infty }\epsilon (t)=0$. In this damped inertial system, the damping coefficient $\frac{\alpha }{t}$ vanishes asymptotically, but not too quickly, a key property to obtain rapid convergence of the values. In the case $\epsilon (\cdot )\equiv 0$, this dynamic has been highlighted recently by Su, Boyd, and Candès as a continuous version of the Nesterov accelerated method. Depending on the speed of convergence of $\epsilon (t)$ to zero, we analyze the convergence properties of the trajectories of $\mbox{(AVD)}_{α, ε}$. We obtain results ranging from the rapid convergence of $\Phi (x(t))$ to $\min \Phi$ when $\epsilon (t)$ decreases rapidly to zero, up to the strong ergodic convergence of the trajectories to the element of minimal norm of the set of minimizers of $\Phi$, when $\epsilon (t)$ tends slowly to zero.