6 papers · 1 filter
Decentralized Inexact Cubic Newton Method with Consensus Procedure
Artem Agafonov, Anton Novitskii, Alexander Rogozin +5
Distributed optimization is widely used in large-scale and privacy-preserving machine learning, where each agent stores a local objective and communicates only with its neighbors i…
Cubic Regularized Newton Method with Variance Reduction for Finite-sum Non-convex Problems
Dmitry Pasechnyuk-Vilensky, Dmitry Kamzolov, Martin TakáÄ
We study finite-sum non-convex optimization and analyze a variance-reduced cubic Newton method based on EMA-smoo…
Adaptive Regularized Newton Method with Inexact Hessian
Aleksandr Shestakov, Nail Bashirov, Andrei Semenov +4
Newton's method is the most widespread high-order method, demanding the gradient and the Hessian of the objective function. However, one of the main disadvantages of Newtons method…
OPTAMI: Global Superlinear Convergence of High-order Methods
Dmitry Kamzolov, Dmitry Pasechnyuk, Artem Agafonov +2
Second-order methods for convex optimization outperform first-order methods in terms of theoretical iteration convergence, achieving rates up to for highly-smooth funct…
Exploring Jacobian Inexactness in Second-Order Methods for Variational Inequalities: Lower Bounds, Optimal Algorithms and Quasi-Newton Approximations
Artem Agafonov, Petr Ostroukhov, Roman Mozhaev +5
Variational inequalities represent a broad class of problems, including minimization and min-max problems, commonly found in machine learning. Existing second-order and high-order…
Advancing the lower bounds: An accelerated, stochastic, second-order method with optimal adaptation to inexactness
Artem Agafonov, Dmitry Kamzolov, Alexander Gasnikov +4
We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theore…