28 citations · 184 across the 34 of their papers we have counts for
48 papers
Apertus 1.5: From Open Text Models to Open Multimodal Reasoners
Project Apertus, :, Yixuan Xu +130
Language models are becoming infrastructure, and infrastructure built in the open, by public institutions, on documented data, is the alternative to dependence on a few providers;…
Adaptive two-points zero-order accelerated methods for stochastic convex optimization problems
Yuriy Dorn, Osman Osmanov, Timofei Loginov +3
We study stochastic convex optimization, where each sampled function can be evaluated at two arbitrary points. Our accelerated method chooses step sizes, batch sizes and smoothing…
Optimal Rates for Federated Variational Inequalities under different relaxations of bounded variance assumption
Kirill Novoselov, Yuriy Dorn, Darina Dvinskikh +1
We study how interpolation affects local steps and parallel sampling in monotone variational inequalities (VIs), with workers, oracle calls per worker per round, and ro…
Minimax Oracle Call Complexity, Rounds and Admissible Noise in Zeroth-Order Optimization of Highly Smooth Functions
Timofei Loginov, Darina Dvinskikh, Nazarii Tupitsa +3
We study zeroth-order optimization of convex and strongly convex functions with Hölder smoothness of order from values corrupted by random noise with bias independent of the…
Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning
Ilya Kuruzov, Dmitrii Vishovan, Kirill Novoselov +3
A federated objective is a weighted sum of client risks, and the weights are almost always fixed in advance. We treat them instead as the only instrument of a wisdom-of-crowds mech…
Lower Bounds For Gradient-Free Convex Optimization And Convex-Concave Saddle-Point Problems
Yuriy Dorn, Darina Dvinskikh, Timofei Loginov +4
We study the query complexity of optimization with exact scalar-value information. For globally -smooth convex functions on with a minimizer in a Euclidean ball of…
Gradient-Free Methods for Stochastic Convex Optimization with Stochastic Functional Constraints
Vadim Abronin, Alexander Gasnikov, Darina Dvinskikh
We develop accelerated gradient-free methods for stochastic convex optimization with constraints defined by expectations. Our batched primal-dual sliding method uses two-point eval…
Accelerated Consensus and Decentralized Optimization over Slowly Varying Networks
Darina Dvinskikh, Anton Novitskii, Demyan Yarmoshik +3
We study average consensus over slowly time-varying networks and its application to decentralized optimization. We introduce WAVE, a windowed Chebyshev method that limits the accum…
Accelerated Stochastic Method under -Smoothness and Heavy-Tailed Noise
Aleksandr Lobanov, Darina Dvinskikh, Alexander Gasnikov
We develop an accelerated stochastic method for convex -smooth optimization under heavy-tailed noise. The unbiased oracle has a finite -th noise moment, , wi…
Clipped or Unclipped? Finite-Sample Trade-offs for Averaged SGD under Heavy-Tailed Noise
Alexandra Suvorikova, Egor Gladin, Darina Dvinskikh +5
Gradient clipping is widely used to stabilize training, but it need not improve the statistical accuracy of averaged SGD, even under heavy-tailed noise. We derive a finite-sample c…
The Complexity of Convex Optimization with Mismatched Geometry
Timofei Loginov, Alexander Gasnikov, Yuriy Dorn +4
Optimal first-order methods on non-Euclidean domains such as the ball pair the prox-function with the norm in which smoothness…
Localize, Restart, Accelerate: Stochastic Optimization under Generalized Smoothness
Darina Dvinskikh, Alexander Gasnikov, Aleksandr Lobanov +1
We study stochastic convex optimization under asymmetric \((L_0,L_1)\)-generalized smoothness, a model motivated by machine-learning objectives whose local curvature may grow with…
Wall-Clock Complexity for Zeroth-Order Optimization with Tunable Oracle Fidelity
Alexandra Suvorikova, Igor Pavlov, Artem Vasin +4
Zeroth-order (black-box) optimization is applied when gradients are unavailable and objective evaluations rely on expensive simulations. In many such applications, the oracle fidel…
Stochastic Optimization and Data Science
Arutyun Avetisyan, Darina Dvinskikh, Alexander Gasnikov +3
This paper aims to motivate stochastic optimization problems from a statistical perspective and a statistical learning perspective, where the goal is to maximize the log-likelihood…
Accelerated zero-order SGD under high-order smoothness and overparameterized regime
Georgii Bychkov, Darina Dvinskikh, Anastasia Antsiferova +2
We present a novel gradient-free algorithm to solve a convex stochastic optimization problem, such as those encountered in medicine, physics, and machine learning (e.g., adversaria…
Gradient-free algorithm for saddle point problems under overparametrization
Ekaterina Statkevich, Sofiya Bondar, Darina Dvinskikh +2
This paper focuses on solving a stochastic saddle point problem (SPP) under an overparameterized regime for the case, when the gradient computation is impractical. As an intermedia…
About some works of Boris Polyak on convergence of gradient methods and their development
Seydamet Ablaev, Aleksandr Beznosikov, Alexander Gasnikov +4
The paper presents a review of the state-of-the-art of subgradient and accelerated methods of convex optimization, including in the presence of disturbances and access to various i…
Bregman Proximal Method for Efficient Communications under Similarity
Aleksandr Beznosikov, Darina Dvinskikh, Dmitry Bylinkin +2
We propose a novel stochastic distributed method for both monotone and strongly monotone variational inequalities with Lipschitz operator and proper convex regularizers arising in…
Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance
Nikita Kornilov, Ohad Shamir, Aleksandr Lobanov +5
In this paper, we consider non-smooth stochastic convex optimization with two function evaluations per round under infinite noise variance. In the classical setting when noise has…
Gradient-Free Methods for Non-Smooth Convex Stochastic Optimization with Heavy-Tailed Noise on Convex Compact
Nikita Kornilov, Alexander Gasnikov, Pavel Dvurechensky +1
We present two easy-to-implement gradient-free/zeroth-order methods to optimize a stochastic non-smooth function accessible only via a black-box. The methods are built upon efficie…
Randomized gradient-free methods in convex optimization
Alexander Gasnikov, Darina Dvinskikh, Pavel Dvurechensky +3
This review presents modern gradient-free methods to solve convex optimization problems. By gradient-free methods, we mean those that use only (noisy) realizations of the objective…
Gradient-Free Federated Learning Methods with and -Randomization for Non-Smooth Convex Stochastic Optimization Problems
Aleksandr Lobanov, Belal Alashqar, Darina Dvinskikh +1
This paper studies non-smooth problems of convex stochastic optimization. Using the smoothing technique based on the replacement of the function value at the considered point by th…
Numerical Methods for Large-Scale Optimal Transport
Nazarii Tupitsa, Pavel Dvurechensky, Darina Dvinskikh +1
The optimal transport (OT) problem is a classical optimization problem having the form of linear programming. Machine learning applications put forward new computational challenges…
On the relations of stochastic convex optimization problems with empirical risk minimization problems on -norm balls
Darina Dvinskikh, Vitali Pirau, Alexander Gasnikov
In this paper, we consider convex stochastic optimization problems arising in machine learning applications (e.g., risk minimization) and mathematical statistics (e.g., maximum lik…