Distributed Training of Graph Convolutional Networks
arXiv:2007.06281 · doi:10.1109/TSIPN.2020.3046237
Abstract
The aim of this work is to develop a fully-distributed algorithmic framework for training graph convolutional networks (GCNs). The proposed method is able to exploit the meaningful relational structure of the input data, which are collected by a set of agents that communicate over a sparse network topology. After formulating the centralized GCN training problem, we first show how to make inference in a distributed scenario where the underlying data graph is split among different agents. Then, we propose a distributed gradient descent procedure to solve the GCN training problem. The resulting model distributes computation along three lines: during inference, during back-propagation, and during optimization. Convergence to stationary solutions of the GCN training problem is also established under mild conditions. Finally, we propose an optimization criterion to design the communication topology between agents in order to match with the graph describing data relationships. A wide set of numerical results validate our proposal. To the best of our knowledge, this is the first work combining graph convolutional neural networks with distributed optimization.
arXiv commentPublished on IEEE Transactions on Signal and Information Processing over Networks
References in corpus (19)
- A Comprehensive Survey on Graph Neural Networks
- Deep Learning with Differential Privacy
- Geometric deep learning: going beyond Euclidean data
- Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
- Constrained Consensus
- Federated Optimization: Distributed Machine Learning for On-Device Intelligence
- Graph Convolutional Matrix Completion
- Signals on Graphs: Uncertainty Principle and Sampling
- A Gentle Introduction to Deep Learning for Graphs
- Autoregressive Moving Average Graph Filtering
- Kernel-based Reconstruction of Graph Signals
- Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networks
- A Parallel Stochastic Approximation Method for Nonconvex Multi-Agent Optimization Problems
- Adaptive Graph Signal Processing: Algorithms and Optimal Sampling Strategies
- Advances in Distributed Graph Filtering
- Hybrid Random/Deterministic Parallel Algorithms for Nonconvex Big Data Optimization
- Distributed Adaptive Learning of Graph Signals
- Adaptation and learning over networks under subspace constraints -- Part I: Stability Analysis
- A Framework for Parallel and Distributed Training of Neural Networks