A Distributed Cubic-Regularized Newton Method for Smooth Convex Optimization over Networks

Abstract

We propose a distributed, cubic-regularized Newton method for large-scale convex optimization over networks. The proposed method requires only local computations and communications and is suitable for federated learning applications over arbitrary network topologies. We show a $O(k^{-3})$ convergence rate when the cost function is convex with Lipschitz gradient and Hessian, with $k$ being the number of iterations. We further provide network-dependent bounds for the communication required in each step of the algorithm. We provide numerical experiments that validate our theoretical results.

arXiv comment22 pages, 2 figures. Preprint, under review

References in corpus (18)

• Communication Efficient Distributed Optimization using an Approximate Newton-type Method
• Optimal algorithms for smooth and strongly convex distributed optimization in networks
• Peer-to-peer Federated Learning on Graphs
• Decentralized Quasi-Newton Methods
• Optimal Algorithms for Non-Smooth Distributed Optimization in Networks
• Multi-consensus Decentralized Accelerated Gradient Descent
• Second-Order Methods with Cubic Regularization Under Inexact Information
• Stochastic Variance-Reduced Cubic Regularized Newton Method
• Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization
• The global rate of convergence for optimal tensor methods in smooth convex optimization
• Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems
• Randomized Block Cubic Newton Method
• A Cubic Regularized Newton's Method over Riemannian Manifolds
• Near-Optimal Hyperfast Second-Order Method for convex optimization and its Sliding
• Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
• A Unified Scheme to Accelerate Adaptive Cubic Regularization and Gradient Methods for Convex Optimization
• A Note on Inexact Condition for Cubic Regularized Newton's Method
• Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization

Cited by in corpus (3)

• Distributed Stochastic Nested Optimization via Cubic Regularization
• Distributed Newton-like Algorithms and Learning for Optimized Power Dispatch
• Newton Method over Networks is Fast up to the Statistical Precision