A Robust Gradient Tracking Method for Distributed Optimization over Directed Networks

Abstract

In this paper, we consider the problem of distributed consensus optimization over multi-agent networks with directed network topology. Assuming each agent has a local cost function that is smooth and strongly convex, the global objective is to minimize the average of all the local cost functions. To solve the problem, we introduce a robust gradient tracking method (R-Push-Pull) adapted from the recently proposed Push-Pull/AB algorithm. R-Push-Pull inherits the advantages of Push-Pull and enjoys linear convergence to the optimal solution with exact communication. Under noisy information exchange, R-Push-Pull is more robust than the existing gradient tracking based algorithms; the solutions obtained by each agent reach a neighborhood of the optimum in expectation exponentially fast under a constant stepsize policy. We provide a numerical example that demonstrate the effectiveness of R-Push-Pull.

References in corpus (10)

• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• Diffusion Adaptation Strategies for Distributed Optimization and Learning over Networks
• A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates
• Optimal algorithms for smooth and strongly convex distributed optimization in networks
• Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication
• An Exact Quantized Decentralized Gradient Descent Algorithm
• Distributed Recursive Least-Squares: Stability and Performance Analysis
• ExtraPush for convex smooth decentralized optimization over directed networks
• A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent
• Optimal Algorithms for Distributed Optimization