5 citations · 10 across the 21 of their papers we have counts for
10 papers · 1 filter
Optimal Parameter-Free Second-order Acceleration with Inexact Hessians under Generalized Smoothness
Artem Agafonov, Aslan Ashabokov, Alexander D'yakonov +3
We study second-order convex optimization under generalized Hessian smoothness, where the local Hessian Lipschitz constant may grow linearly with the gradient norm. For inexact Hes…
Certified Residual Quasi-Newton Methods for Distributed Variational Inequalities
Ewsey R. Obzherin, Roman M. Mozhaev, Alexander V. Gasnikov +3
Second-order methods for smooth monotone variational inequalities reach the optimal rate , but a distributed exact Jacobian costs times more communication than an…
Application of Optimal Inexact Second-Order Acceleration to Distributed Stochastic Optimization under Statistical Similarity
Yury A. Sokolov, Maxim K. Mashtaler, Alexander V. Gasnikov +3
We consider distributed stochastic convex optimization with a fixed budget of independent samples split among workers. Sample average approximation reduces the problem to a…
Chebyshev-Exact Acceleration under Hessian Variation, I: Sine-Jacobi Method
Dmitry Pasechnyuk-Vilensky, Martin Takáč
We study finite-horizon one-gradient realizations with the Chebyshev minimax terminal residual on . Under time-dependent Hessian perturbations, the terminal first variation…
Decentralized Inexact Cubic Newton Method with Consensus Procedure
Artem Agafonov, Anton Novitskii, Alexander Rogozin +5
Distributed optimization is widely used in large-scale and privacy-preserving machine learning, where each agent stores a local objective and communicates only with its neighbors i…
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by u…
OPTAMI: Global Superlinear Convergence of High-order Methods
Dmitry Kamzolov, Dmitry Pasechnyuk, Artem Agafonov +2
Second-order methods for convex optimization outperform first-order methods in terms of theoretical iteration convergence, achieving rates up to for highly-smooth funct…
Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities
Aleksandr Beznosikov, Martin Takáč, Alexander Gasnikov
Variational inequalities are a broad and flexible class of problems that includes minimization, saddle point, and fixed point problems as special cases. Therefore, variational ineq…
FLECS: A Federated Learning Second-Order Framework via Compression and Sketching
Artem Agafonov, Dmitry Kamzolov, Rachael Tappenden +2
Inspired by the recent work FedNL (Safaryan et al, FedNL: Making Newton-Type Methods Applicable to Federated Learning), we propose a new communication efficient second-order framew…
Inexact Tensor Methods and Their Application to Stochastic Convex Optimization
Artem Agafonov, Dmitry Kamzolov, Pavel Dvurechensky +2
We propose general non-accelerated and accelerated tensor methods under inexact information on the derivatives of the objective, analyze their convergence rate. Further, we provide…