Efficient Graph Generation with Graph Recurrent Attention Networks

Abstract

We propose a new family of efficient and expressive deep generative models of graphs, called Graph Recurrent Attention Networks (GRANs). Our model generates graphs one block of nodes and associated edges at a time. The block size and sampling stride allow us to trade off sample quality for efficiency. Compared to previous RNN-based graph generative models, our framework better captures the auto-regressive conditioning between the already-generated and to-be-generated parts of the graph using Graph Neural Networks (GNNs) with attention. This not only reduces the dependency on node ordering but also bypasses the long-term bottleneck caused by the sequential nature of RNNs. Moreover, we parameterize the output distribution per block using a mixture of Bernoulli, which captures the correlations among generated edges within the block. Finally, we propose to handle node orderings in generation by marginalizing over a family of canonical orderings. On standard benchmarks, we achieve state-of-the-art time efficiency and sample quality compared to previous models. Additionally, we show our model is capable of generating large graphs of up to 5K nodes with good quality. To the best of our knowledge, GRAN is the first deep graph generative model that can scale to this size. Our code is released at: https://github.com/lrjconan/GRAN.

arXiv commentNeural Information Processing Systems (NeurIPS) 2019

Cited by in corpus (53)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings
• Keeping it Simple: Language Models can learn Complex Molecular Distributions
• Molecular graph generation with Graph Neural Networks
• Hierarchical Generation of Molecular Graphs using Structural Motifs
• GPT-GNN: Generative Pre-Training of Graph Neural Networks
• Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design
• PolyGen: An Autoregressive Generative Model of 3D Meshes
• On the Binding Problem in Artificial Neural Networks
• Motif-based Graph Self-Supervised Learning for Molecular Property Prediction
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Image-Conditioned Graph Generation for Road Network Extraction
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• House-GAN++: Generative Adversarial Layout Refinement Networks
• TIGGER: Scalable Generative Modelling for Temporal Interaction Graphs
• Goal-directed graph construction using reinforcement learning
• Graph Coarsening with Neural Networks
• Categorical Normalizing Flows via Continuous Transformations
• Score-based Generative Modeling in Latent Space
• Graph Deconvolutional Generation
• Deception for Cyber Defence: Challenges and Opportunities
• Latent Variable Modelling with Hyperbolic Normalizing Flows
• Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning
• Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences
• Deep Generative Models to Extend Active Directory Graphs with Honeypot Users
• SCGG: A Deep Structure-Conditioned Graph Generative Model
• E(n) Equivariant Normalizing Flows
• Graph-Aware Transformer: Is Attention All Graphs Need?
• SceneGen: Learning to Generate Realistic Traffic Scenes
• KuraNet: Systems of Coupled Oscillators that Learn to Synchronize
• Non-Parametric Graph Learning for Bayesian Graph Neural Networks
• DeepGG: a Deep Graph Generator
• CCGG: A Deep Autoregressive Model for Class-Conditional Graph Generation
• Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation
• DeepNC: Deep Generative Network Completion
• Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions
• Deep Generative Modeling in Network Science with Applications to Public Policy Research
• Auto-decoding Graphs
• Generating a Doppelganger Graph: Resembling but Distinct
• Top-N: Equivariant set and graph generation without exchangeability
• Partition and Code: learning how to compress graphs
• SchemaDB: Structures in Relational Datasets
• On the Power of Edge Independent Graph Models
• Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• Generating the Graph Gestalt: Kernel-Regularized Graph Representation Learning
• X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering
• Cyclic Graph Attentive Match Encoder (CGAME): A Novel Neural Network For OD Estimation
• Multi-Glimpse Network: A Robust and Efficient Classification Architecture based on Recurrent Downsampled Attention
• TD-GEN: Graph Generation With Tree Decomposition
• Isometric Graph Neural Networks
• Hierarchical BiGraph Neural Network as Recommendation Systems
• ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning