Graph Balancing for Distributed Subgradient Methods over Directed Graphs
arXiv:1603.03461
Abstract
We consider a multi agent optimization problem where a set of agents collectively solves a global optimization problem with the objective function given by the sum of locally known convex functions. We focus on the case when information exchange among agents takes place over a directed network and propose a distributed subgradient algorithm in which each agent performs local processing based on information obtained from his incoming neighbors. Our algorithm uses weight balancing to overcome the asymmetries caused by the directed communication network, i.e., agents scale their outgoing information with dynamically updated weights that converge to balancing weights of the graph. We show that both the objective function values and the consensus violation, at the ergodic average of the estimates generated by the algorithm, converge with rate , where is the number of iterations. A special case of our algorithm provides a new distributed method to compute average consensus over directed graphs.
References in corpus (7)
- Constrained Consensus
- HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
- Diffusion Adaptation Strategies for Distributed Optimization and Learning over Networks
- Optimal Distributed Online Prediction using Mini-Batches
- On the Convergence of Decentralized Gradient Descent
- Optimality guarantees for distributed statistical estimation
- EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization