Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks

Abstract

Advancing research in the emerging field of deep graph learning requires new tools to support tensor computation over graphs. In this paper, we present the design principles and implementation of Deep Graph Library (DGL). DGL distills the computational patterns of GNNs into a few generalized sparse tensor operations suitable for extensive parallelization. By advocating graph as the central programming abstraction, DGL can perform optimizations transparently. By cautiously adopting a framework-neutral design, DGL allows users to easily port and leverage the existing components across multiple deep learning frameworks. Our evaluation shows that DGL significantly outperforms other popular GNN-oriented frameworks in both speed and memory consumption over a variety of benchmarks and has little overhead for small scale workloads.

arXiv commentMajor update with significantly more results

References in corpus (24)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Spectral Networks and Locally Connected Networks on Graphs
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Modeling polypharmacy side effects with graph convolutional networks
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Fast Graph Representation Learning with PyTorch Geometric
• Simplifying Graph Convolutional Networks
• Graph Convolutional Matrix Completion
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Learning Deep Generative Models of Graphs
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• How Powerful are Graph Neural Networks?
• Stochastic Training of Graph Convolutional Networks with Variance Reduction
• Supervised Community Detection with Line Graph Neural Networks
• Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs
• Embedding Logical Queries on Knowledge Graphs
• GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
• End-to-End Goal-Driven Web Navigation
• Fast Sparse Matrix-Vector Multiplication on GPUs: Implications for Graph Mining
• Towards Efficient Large-Scale Graph Neural Network Computing

Cited by in corpus (143)

• Recent Advances and Applications of Deep Learning Methods in Materials Science
• Atomistic Line Graph Neural Network for Improved Materials Property Predictions
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• DGL-LifeSci: An Open-Source Toolkit for Deep Learning on Graphs in Life Science
• Combinatorial Optimization with Physics-Inspired Graph Neural Networks
• A Review of Biomedical Datasets Relating to Drug Discovery: A Knowledge Graph Perspective
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Multi-Robot Collaborative Perception with Graph Neural Networks
• Advances of Machine Learning in Materials Science: Ideas and Techniques
• End-to-End Differentiable Molecular Mechanics Force Field Construction
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• 3D Infomax improves GNNs for Molecular Property Prediction
• Graph Autoencoders for Embedding Learning in Brain Networks and Major Depressive Disorder Identification
• Rethinking Graph Transformers with Spectral Attention
• Automated Machine Learning on Graphs: A Survey
• Actionable and Interpretable Fault Localization for Recurring Failures in Online Service Systems
• KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction
• Graph Neural Networks Accelerated Molecular Dynamics
• Gibbs-Duhem-Informed Neural Networks for Binary Activity Coefficient Prediction
• DIG: A Turnkey Library for Diving into Graph Deep Learning Research
• Domain Adversarial Graph Convolutional Network Based on RSSI and Crowdsensing for Indoor Localization
• SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization
• Machine-learned molecular mechanics force field for the simulation of protein-ligand systems and beyond
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models
• How Attentive are Graph Attention Networks?
• Learning the Travelling Salesperson Problem Requires Rethinking Generalization
• Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Caching
• HP-GNN: Generating High Throughput GNN Training Implementation on CPU-FPGA Heterogeneous Platform
• Relational Graph Convolutional Networks: A Closer Look
• Recent progress in the JARVIS infrastructure for next-generation data-driven materials design
• Drug-Target Interaction/Affinity Prediction: Deep Learning Models and Advances Review
• Quarks and gluons in the Lund plane
• Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from Different Chemical Spaces
• Few-shot link prediction via graph neural networks for Covid-19 drug-repurposing
• On the design space between molecular mechanics and machine learning force fields
• A Deep-learning Model for Fast Prediction of Vacancy Formation in Diverse Materials
• AGL: a Scalable System for Industrial-purpose Graph Machine Learning
• Grappa -- A Machine Learned Molecular Mechanics Force Field
• Bag of Tricks for Node Classification with Graph Neural Networks
• Interpreting and Unifying Graph Neural Networks with An Optimization Framework
• Implementing graph neural networks with TensorFlow-Keras
• Subgroup Generalization and Fairness of Graph Neural Networks
• Scalable Graph Neural Networks for Heterogeneous Graphs
• Gaussian Process Molecule Property Prediction with FlowMO
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• A community-powered search of machine learning strategy space to find NMR property prediction models
• DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs
• IGNNITION: Bridging the Gap Between Graph Neural Networks and Networking Systems
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining
• Graph Neural Networks and Representation Embedding for Table Extraction in PDF Documents
• Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training
• Local Augmentation for Graph Neural Networks
• A Network Science perspective of Graph Convolutional Networks: A survey
• KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs
• Hierarchical Interdisciplinary Topic Detection Model for Research Proposal Classification
• CGNN: Traffic Classification with Graph Neural Network
• Automatic selection of clustering algorithms using supervised graph embedding
• Improved Constraints on Effective Top Quark Interactions using Edge Convolution Networks
• TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion
• Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
• Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practices
• AtomVision: A machine vision library for atomistic images
• Modeling User Behavior With Interaction Networks for Spam Detection
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering
• Optimal Solving of Constrained Path-Planning Problems with Graph Convolutional Networks and Optimized Tree Search
• Graph Neural Networks Inspired by Classical Iterative Algorithms
• FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems
• p2pGNN: A Decentralized Graph Neural Network for Node Classification in Peer-to-Peer Networks
• Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit
• Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages
• Repurpose Open Data to Discover Therapeutics for COVID-19 using Deep Learning
• You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks
• LibAUC: A Deep Learning Library for X-Risk Optimization
• Graph Neural Network Guided Local Search for the Traveling Salesperson Problem
• Reflection-Equivariant Diffusion for 3D Structure Determination from Isotopologue Rotational Spectra in Natural Abundance
• Hybrid Micro/Macro Level Convolution for Heterogeneous Graph Learning
• Towards Representation Identical Privacy-Preserving Graph Neural Network via Split Learning
• Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation
• Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
• Similar Cases Recommendation using Legal Knowledge Graphs
• Network In Graph Neural Network
• Adversarial Attacks on Graph Classification via Bayesian Optimisation
• Multi-Graph based Multi-Scenario Recommendation in Large-scale Online Video Services
• FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
• Coarse-grained crystal graph neural networks for reticular materials design
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery
• Calibrating and Improving Graph Contrastive Learning
• Learning Coordination Policies over Heterogeneous Graphs for Human-Robot Teams via Recurrent Neural Schedule Propagation
• A Baselined Gated Attention Recurrent Network for Request Prediction in Ridesharing
• Exploring the Representational Power of Graph Autoencoder
• Communication-Efficient Sampling for Distributed Training of Graph Convolutional Networks
• Scalable Power Control/Beamforming in Heterogeneous Wireless Networks with Graph Neural Networks
• SCV-GNN: Sparse Compressed Vector-based Graph Neural Network Aggregation
• CogDL: A Comprehensive Library for Graph Deep Learning
• Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis
• DeSCo: Towards Generalizable and Scalable Deep Subgraph Counting
• Covid-19 Detection from Chest X-ray and Patient Metadata using Graph Convolutional Neural Networks
• Heterogeneous Graph Attention Networks for Learning Diverse Communication
• Systematic Generalization on gSCAN with Language Conditioned Embedding
• Graph convolutional network as a fast statistical emulator for numerical ice sheet modeling
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• Jet tagging in the Lund plane with graph networks
• GraphGallery: A Platform for Fast Benchmarking and Easy Development of Graph Neural Networks Based Intelligent Software
• Graph Neural Network-based Surrogate Models for Finite Element Analysis
• Weakly Supervised Concept Map Generation through Task-Guided Graph Translation
• Learning to Pool in Graph Neural Networks for Extrapolation
• Graph Inference Representation: Learning Graph Positional Embeddings with Anchor Path Encoding
• Accurate Prediction of Free Solvation Energy of Organic Molecules via Graph Attention Network and Message Passing Neural Network from Pairwise Atomistic Interactions
• Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
• Mesorasi: Architecture Support for Point Cloud Analytics via Delayed-Aggregation
• PyTorch-Direct: Enabling GPU Centric Data Access for Very Large Graph Neural Network Training with Irregular Accesses
• Lossless Compression of Structured Convolutional Models via Lifting
• gSuite: A Flexible and Framework Independent Benchmark Suite for Graph Neural Network Inference on GPUs
• Learning Hierarchical Graph Neural Networks for Image Clustering
• A Framework for Joint Unsupervised Learning of Cluster-Aware Embedding for Heterogeneous Networks
• SoGCN: Second-Order Graph Convolutional Networks
• Should Graph Neural Networks Use Features, Edges, Or Both?
• Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning
• GNNIE: GNN Inference Engine with Load-balancing and Graph-Specific Caching
• Convergent Graph Solvers
• Spotting Silent Buffer Overflows in Execution Trace through Graph Neural Network Assisted Data Flow Analysis
• TyXe: Pyro-based Bayesian neural nets for Pytorch
• On Graph Neural Network Ensembles for Large-Scale Molecular Property Prediction
• What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks
• Analyzing the Performance of Graph Neural Networks with Pipe Parallelism
• Learning Co-Speech Gesture for Multimodal Aphasia Type Detection
• Global Neighbor Sampling for Mixed CPU-GPU Training on Giant Graphs
• Document Structure aware Relational Graph Convolutional Networks for Ontology Population
• PanRep: Graph neural networks for extracting universal node embeddings in heterogeneous graphs
• Efficient Sparse Matrix Kernels based on Adaptive Workload-Balancing and Parallel-Reduction
• Learning Large-Time-Step Molecular Dynamics with Graph Neural Networks
• Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features
• TME-BNA: Temporal Motif-Preserving Network Embedding with Bicomponent Neighbor Aggregation
• ICLR 2021 Challenge for Computational Geometry & Topology: Design and Results
• Graph Conditioned Sparse-Attention for Improved Source Code Understanding
• Graph Neural Network for Crawling Target Nodes in Social Networks
• Resource Allocation in Disaggregated Data Centre Systems with Reinforcement Learning
• Graph Deep Factors for Forecasting
• Graph Neural Network Training with Data Tiering
• Leveraging Tripartite Interaction Information from Live Stream E-Commerce for Improving Product Recommendation