Nested Distributed Gradient Methods with Adaptive Quantized Communication
arXiv:1903.08149
Abstract
In this paper, we consider minimizing a sum of local convex objective functions in a distributed setting, where communication can be costly. We propose and analyze a class of nested distributed gradient methods with adaptive quantized communication (NEAR-DGD+Q). We show the effect of performing multiple quantized communication steps on the rate of convergence and on the size of the neighborhood of convergence, and prove R-Linear convergence to the exact solution with increasing number of consensus steps and adaptive quantization. We test the performance of the method, as well as some practical variants, on quadratic functions, and show the effects of multiple quantized communication steps in terms of iterations/gradient evaluations, communication and cost.
arXiv comment9 pages, 2 figures. arXiv admin note: text overlap with arXiv:1709.02999
References in corpus (7)
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- Communication Efficient Distributed Optimization using an Approximate Newton-type Method
- Distributed Learning in Wireless Sensor Networks
- A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates
- An Exact Quantized Decentralized Gradient Descent Algorithm
- Communication/Computation Tradeoffs in Consensus-Based Distributed Optimization
- Distributed Stochastic Approximation for Solving Network Optimization Problems Under Random Quantization