7 papers
A Few Accelerated Algorithms for Convex Optimization under -Smoothness
Aleksandr Lobanov
We develop accelerated algorithms for convex -smooth optimization, where . This class generalizes standard smoothness and contain…
Avoiding Bias in Clipped SGD for Overparameterized Models under Generalized Smoothness
Aleksandr Lobanov, Anastasia Koloskova
Modern machine learning is dominated by complex, overparameterized architectures capable of interpolating data and achieving zero training loss. For such models, we investigate the…
Median Clipping for Zeroth-order Non-Smooth Convex Optimization and Multi-Armed Bandit Problem with Heavy-tailed Symmetric Noise
Nikita Kornilov, Yuriy Dorn, Aleksandr Lobanov +5
In this paper, we consider non-smooth convex optimization with a zeroth-order oracle corrupted by symmetric stochastic noise. Unlike the existing high-probability results requiring…
Power of Generalized Smoothness in Stochastic Convex Optimization: First- and Zero-Order Algorithms
Aleksandr Lobanov, Alexander Gasnikov
This paper is devoted to the study of stochastic optimization problems under the generalized smoothness assumption. By considering the unbiased gradient oracle in Stochastic Gradie…
Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under -Smoothness
Aleksandr Lobanov, Alexander Gasnikov, Eduard Gorbunov +1
The gradient descent (GD) method -- is a fundamental and likely the most popular optimization algorithm in machine learning (ML), with a history traced back to a paper in 1847 (Cau…
Acceleration Exists! Optimization Problems When Oracle Can Only Compare Objective Function Values
Aleksandr Lobanov, Alexander Gasnikov, Andrei Krasnov
Frequently, the burgeoning field of black-box optimization encounters challenges due to a limited understanding of the mechanisms of the objective function. To address such problem…