FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems

Abstract

Graph neural networks (GNNs) are gaining increasing popularity as a promising approach to machine learning on graphs. Unlike traditional graph workloads where each vertex/edge is associated with a scalar, GNNs attach a feature tensor to each vertex/edge. This additional feature dimension, along with consequently more complex vertex- and edge-wise computations, has enormous implications on locality and parallelism, which existing graph processing systems fail to exploit. This paper proposes FeatGraph to accelerate GNN workloads by co-optimizing graph traversal and feature dimension computation. FeatGraph provides a flexible programming interface to express diverse GNN models by composing coarse-grained sparse templates with fine-grained user-defined functions (UDFs) on each vertex/edge. FeatGraph incorporates optimizations for graph traversal into the sparse templates and allows users to specify optimizations for UDFs with a feature dimension schedule (FDS). FeatGraph speeds up end-to-end GNN training and inference by up to 32x on CPU and 7x on GPU.

SC'20; changed all figures to type 1

References in corpus (7)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Fast Graph Representation Learning with PyTorch Geometric
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• Attention-based Graph Neural Network for Semi-supervised Learning
• Deep Representation Learning for Social Network Analysis

Cited by in corpus (3)

• Ansor: Generating High-Performance Tensor Programs for Deep Learning
• Hector: An Efficient Programming and Compilation Framework for Implementing Relational Graph Neural Networks in GPU Architectures
• Efficient Sparse Matrix Kernels based on Adaptive Workload-Balancing and Parallel-Reduction