HP-GNN: Generating High Throughput GNN Training Implementation on CPU-FPGA Heterogeneous Platform

Abstract

Graph Neural Networks (GNNs) have shown great success in many applications such as recommendation systems, molecular property prediction, traffic prediction, etc. Recently, CPU-FPGA heterogeneous platforms have been used to accelerate many applications by exploiting customizable data path and abundant user-controllable on-chip memory resources of FPGAs. Yet, accelerating and deploying GNN training on such platforms requires not only expertise in hardware design but also substantial development efforts. We propose HP-GNN, a novel framework that generates high throughput GNN training implementations on a given CPU-FPGA platform that can benefit both application developers and machine learning researchers. HP-GNN takes GNN training algorithms, GNN models as the inputs, and automatically performs hardware mapping onto the target CPU-FPGA platform. HP-GNN consists of: (1) data layout and internal representation that reduce the memory traffic and random memory accesses; (2) optimized hardware templates that support various GNN models; (3) a design space exploration engine for automatic hardware mapping; (4) high-level application programming interfaces (APIs) that allows users to specify GNN training with only a handful of lines of code. To evaluate HP-GNN, we experiment with two well-known sampling-based GNN training algorithms and two GNN models. For each training algorithm and model, HP-GNN generates implementation on a state-of-the-art CPU-FPGA platform. Compared with CPU-only and CPU-GPU platforms, experimental results show that the generated implementations achieve $55.67\times$ and $2.17\times$ speedup on the average, respectively. Compared with the state-of-the-art GNN training implementations, HP-GNN achieves up to $4.45\times$ speedup.

References in corpus (9)

• TensorFlow: A system for large-scale machine learning
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Graph Neural Network for Traffic Forecasting: A Survey
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• Stochastic Training of Graph Convolutional Networks with Variance Reduction
• GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms

Cited by in corpus (2)

• Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA
• GNNBuilder: An Automated Framework for Generic Graph Neural Network Accelerator Generation, Simulation, and Optimization