Analyzing the Performance of Graph Neural Networks with Pipe Parallelism
arXiv:2012.10840
Abstract
Many interesting datasets ubiquitous in machine learning and deep learning can be described via graphs. As the scale and complexity of graph-structured datasets increase, such as in expansive social networks, protein folding, chemical interaction networks, and material phase transitions, improving the efficiency of the machine learning techniques applied to these is crucial. In this study, we focus on Graph Neural Networks (GNN) that have found great success in tasks such as node or edge classification and link prediction. However, standard GNN models have scaling limits due to necessary recursive calculations performed through dense graph relationships that lead to memory and runtime bottlenecks. While new approaches for processing larger networks are needed to advance graph techniques, and several have been proposed, we study how GNNs could be parallelized using existing tools and frameworks that are known to be successful in the deep learning community. In particular, we investigate applying pipeline parallelism to GNN models with GPipe, introduced by Google in 2018.
Proceedings of the conference MLSys'21 Workshop on Graph Neural Networks and Systems (GNNSys'21), San Jose, CA, USA, 2021
References in corpus (6)
- Semi-Supervised Classification with Graph Convolutional Networks
- Cooperative Game Theory Approaches for Network Partitioning
- Deep Convolutional Networks on Graph-Structured Data
- Fast Graph Representation Learning with PyTorch Geometric
- torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models
- A Survey on Graph Neural Networks for Knowledge Graph Completion