4 papers
On the Convergence Analysis of Aggregated Heavy-Ball Method
Marina Danilova
Momentum first-order optimization methods are the workhorses in various optimization tasks, e.g., in the training of deep neural networks. Recently, Lucas et al. (2019) proposed a…
Distributed Methods with Absolute Compression and Error Compensation
Marina Danilova, Eduard Gorbunov
Distributed optimization methods are often applied to solving huge-scale problems like training neural networks with millions and even billions of parameters. In such applications,…
Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping
Eduard Gorbunov, Marina Danilova, Alexander Gasnikov
In this paper, we propose a new accelerated stochastic first-order method called clipped-SSTM for smooth convex stochastic optimization with heavy-tailed distributed noise in stoch…
Non-monotone Behavior of the Heavy Ball Method
Marina Danilova, Anastasiya Kulakova, Boris Polyak
We focus on the solutions of second-order stable linear difference equations and demonstrate that their behavior can be non-monotone and exhibit peak effects depending on initial c…