3 citations · 10 across the 19 of their papers we have counts for
27 papers
Apertus 1.5: From Open Text Models to Open Multimodal Reasoners
Project Apertus, :, Yixuan Xu +130
Language models are becoming infrastructure, and infrastructure built in the open, by public institutions, on documented data, is the alternative to dependence on a few providers;…
Adaptive two-points zero-order accelerated methods for stochastic convex optimization problems
Yuriy Dorn, Osman Osmanov, Timofei Loginov +3
We study stochastic convex optimization, where each sampled function can be evaluated at two arbitrary points. Our accelerated method chooses step sizes, batch sizes and smoothing…
Optimal Rates for Federated Variational Inequalities under different relaxations of bounded variance assumption
Kirill Novoselov, Yuriy Dorn, Darina Dvinskikh +1
We study how interpolation affects local steps and parallel sampling in monotone variational inequalities (VIs), with workers, oracle calls per worker per round, and ro…
Minimax Oracle Call Complexity, Rounds and Admissible Noise in Zeroth-Order Optimization of Highly Smooth Functions
Timofei Loginov, Darina Dvinskikh, Nazarii Tupitsa +3
We study zeroth-order optimization of convex and strongly convex functions with Hölder smoothness of order from values corrupted by random noise with bias independent of the…
Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning
Ilya Kuruzov, Dmitrii Vishovan, Kirill Novoselov +3
A federated objective is a weighted sum of client risks, and the weights are almost always fixed in advance. We treat them instead as the only instrument of a wisdom-of-crowds mech…
Accelerated Stochastic Method under -Smoothness and Heavy-Tailed Noise
Aleksandr Lobanov, Darina Dvinskikh, Alexander Gasnikov
We develop an accelerated stochastic method for convex -smooth optimization under heavy-tailed noise. The unbiased oracle has a finite -th noise moment, , wi…
Optimal Parameter-Free Second-order Acceleration with Inexact Hessians under Generalized Smoothness
Artem Agafonov, Aslan Ashabokov, Alexander D'yakonov +3
We study second-order convex optimization under generalized Hessian smoothness, where the local Hessian Lipschitz constant may grow linearly with the gradient norm. For inexact Hes…
Smooth or Separable? Sparse Block Acceleration for Entropy-Regularized Linear Programs
Irina Podlipnova, Maxim Mashtaler, Artem Agafonov +2
We study entropy-regularized linear programs with sparse affine constraints and compare two exact dual representations induced by whether a redundant normalization constraint is re…
Localize, Restart, Accelerate: Stochastic Optimization under Generalized Smoothness
Darina Dvinskikh, Alexander Gasnikov, Aleksandr Lobanov +1
We study stochastic convex optimization under asymmetric \((L_0,L_1)\)-generalized smoothness, a model motivated by machine-learning objectives whose local curvature may grow with…
Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
Artem Agafonov, Vladislav Ryspayev, Samuel Horváth +3
Quasi-Newton methods are widely used for solving convex optimization problems due to their ease of implementation, practical efficiency, and strong local convergence guarantees. Ho…
Norm-Constrained Flows and Sign-Based Optimization: Theory and Algorithms
Valentin Leplat, Sergio Mayorga, Roland Hildebrand +1
Sign Gradient Descent (SignGD) uses only the coordinate-wise sign of the gradient. We study this method through norm-constrained continuous-time dynamics: at each point, the veloci…
Decentralised convex optimisation with probability-proportional-to-size quantization
Dmitrii Pasechniuk, Pavel Dvurechensky, César A. Uribe +1
Communication is one of the bottlenecks of distributed optimisation and learning. To overcome this bottleneck, we propose a novel quantization method that transforms a vector into…
Power of Generalized Smoothness in Stochastic Convex Optimization: First- and Zero-Order Algorithms
Aleksandr Lobanov, Alexander Gasnikov
This paper is devoted to the study of stochastic optimization problems under the generalized smoothness assumption. By considering the unbiased gradient oracle in Stochastic Gradie…
Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under -Smoothness
Aleksandr Lobanov, Alexander Gasnikov, Eduard Gorbunov +1
The gradient descent (GD) method -- is a fundamental and likely the most popular optimization algorithm in machine learning (ML), with a history traced back to a paper in 1847 (Cau…
Accelerated zero-order SGD under high-order smoothness and overparameterized regime
Georgii Bychkov, Darina Dvinskikh, Anastasia Antsiferova +2
We present a novel gradient-free algorithm to solve a convex stochastic optimization problem, such as those encountered in medicine, physics, and machine learning (e.g., adversaria…
OPTAMI: Global Superlinear Convergence of High-order Methods
Dmitry Kamzolov, Dmitry Pasechnyuk, Artem Agafonov +2
Second-order methods for convex optimization outperform first-order methods in terms of theoretical iteration convergence, achieving rates up to for highly-smooth funct…
Nesterov's method of dichotomy via Order Oracle: The problem of optimizing a two-variable function on a square
Boris Chervonenkis, Andrei Krasnov, Alexander Gasnikov +1
The challenges of black box optimization arise due to imprecise responses and limited output information. This article describes new results on optimizing multivariable functions u…
Gradient-free algorithm for saddle point problems under overparametrization
Ekaterina Statkevich, Sofiya Bondar, Darina Dvinskikh +2
This paper focuses on solving a stochastic saddle point problem (SPP) under an overparameterized regime for the case, when the gradient computation is impractical. As an intermedia…
Acceleration Exists! Optimization Problems When Oracle Can Only Compare Objective Function Values
Aleksandr Lobanov, Alexander Gasnikov, Andrei Krasnov
Frequently, the burgeoning field of black-box optimization encounters challenges due to a limited understanding of the mechanisms of the objective function. To address such problem…
About some works of Boris Polyak on convergence of gradient methods and their development
Seydamet Ablaev, Aleksandr Beznosikov, Alexander Gasnikov +4
The paper presents a review of the state-of-the-art of subgradient and accelerated methods of convex optimization, including in the presence of disturbances and access to various i…
The Black-Box Optimization Problem: Zero-Order Accelerated Stochastic Method via Kernel Approximation
Aleksandr Lobanov, Nail Bashirov, Alexander Gasnikov
In this paper, we study the standard formulation of an optimization problem when the computation of gradient is not available. Such a problem can be classified as a "black box" opt…
Convergence analysis of stochastic gradient descent with adaptive preconditioning for non-convex and convex functions
Dmitrii A. Pasechnyuk, Alexander Gasnikov, Martin Takáč
Preconditioning is a crucial operation in gradient-based numerical optimisation. It helps decrease the local condition number of a function by appropriately transforming its gradie…
Accelerated Zero-Order SGD Method for Solving the Black Box Optimization Problem under "Overparametrization" Condition
Aleksandr Lobanov, Alexander Gasnikov
This paper is devoted to solving a convex stochastic optimization problem in a overparameterization setup for the case where the original gradient computation is not available, but…
Information-Theoretic Upper Bounds for Deterministic Noise in Zeroth-Order Convex Optimization
Dmitry Pasechnyuk-Vilensky, Igor Pavlov, Martin Takáč +1
We study zeroth-order convex optimization on Euclidean balls with function values corrupted by a fixed, uniformly bounded deterministic perturbation. For a query budget , the ma…