Learning Deep Generative Models of Graphs

Abstract

Graphs are fundamental data structures which concisely capture the relational structure in many important real-world domains, such as knowledge graphs, physical and social interactions, language, and chemistry. Here we introduce a powerful new approach for learning generative models over graphs, which can capture both their structure and attributes. Our approach uses graph neural networks to express probabilistic dependencies among a graph's nodes and edges, and can, in principle, learn distributions over any arbitrary graph. In a series of experiments our results show that once trained, our models can generate good quality samples of both synthetic graphs as well as real molecular graphs, both unconditionally and conditioned on data. Compared to baselines that do not use graph-structured representations, our models often perform far better. We also explore key challenges of learning generative models of graphs, such as how to handle symmetries and ordering of elements during the graph generation process, and offer possible solutions. Our work is the first and most general approach for learning generative models over arbitrary graphs, and opens new directions for moving away from restrictions of vector- and sequence-like knowledge representations, toward more expressive and flexible relational data structures.

arXiv comment21 pages

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Emergence of scaling in random networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Message Passing for Quantum Chemistry
• Deep Convolutional Networks on Graph-Structured Data
• Kronecker Graphs: An Approach to Modeling Networks
• Interaction Networks for Learning about Objects, Relations and Physics
• Grammar as a Foreign Language
• SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules
• Order Matters: Sequence to sequence for sets
• Structured Generative Models of Natural Source Code
• Grammar Variational Autoencoder
• Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks
• Molecular Generation with Recurrent Neural Networks (RNNs)

Cited by in corpus (165)

• Graph Neural Networks: A Review of Methods and Applications
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Deep learning for molecular design - a review of the state of the art
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• MolGAN: An implicit generative model for small molecular graphs
• Constrained Graph Variational Autoencoders for Molecule Design
• A Gentle Introduction to Deep Learning for Graphs
• How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey
• Autonomous discovery in the chemical sciences part II: Outlook
• Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
• Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings
• Metrics for Graph Comparison: A Practitioner's Guide
• Keeping it Simple: Language Models can learn Complex Molecular Distributions
• LanczosNet: Multi-Scale Deep Graph Convolutional Networks
• Learning Discrete Structures for Graph Neural Networks
• Molecular graph generation with Graph Neural Networks
• Deep Learning on Graphs: A Survey
• Graph Neural Networks: Taxonomy, Advances and Trends
• Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• Differentiable Graph Module (DGM) for Graph Convolutional Networks
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Hierarchical Generation of Molecular Graphs using Structural Motifs
• Generative Models for Automatic Chemical Design
• Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• D-VAE: A Variational Autoencoder for Directed Acyclic Graphs
• Multilevel Graph Matching Networks for Deep Graph Similarity Learning
• Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning
• Representation Learning for Natural Language Processing
• Deeply learning molecular structure-property relationships using attention- and gate-augmented graph convolutional network
• Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
• A Graph to Graphs Framework for Retrosynthesis Prediction
• Generative Code Modeling with Graphs
• Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design
• Graph Neural Ordinary Differential Equations
• Deep Set Prediction Networks
• StructureNet: Hierarchical Graph Networks for 3D Shape Generation
• MolecularRNN: Generating realistic molecular graphs with optimized properties
• A Two-Step Graph Convolutional Decoder for Molecule Generation
• Learning to Make Generalizable and Diverse Predictions for Retrosynthesis
• Constrained Bayesian Optimization for Automatic Chemical Design
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• BertGCN: Transductive Text Classification by Combining GCN and BERT
• All SMILES Variational Autoencoder
• Graph Normalizing Flows
• Spectral-based Graph Convolutional Network for Directed Graphs
• Graph Representation Learning for Road Type Classification
• Learning to Extend Molecular Scaffolds with Structural Motifs
• SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Design
• Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs
• CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models
• Graphite: Iterative Generative Modeling of Graphs
• Fast Graph Generation via Spectral Diffusion
• Directed Acyclic Graph Neural Networks
• Edge-based sequential graph generation with recurrent neural networks
• Graph Neural Networks: Methods, Applications, and Opportunities
• Multi-Objective De Novo Drug Design with Conditional Graph Generative Model
• Decoding Molecular Graph Embeddings with Reinforcement Learning
• Hierarchical Graph-to-Graph Translation for Molecules
• Implementing graph neural networks with TensorFlow-Keras
• Image-Conditioned Graph Generation for Road Network Extraction
• Learning Interpretable Representations of Entanglement in Quantum Optics Experiments using Deep Generative Models
• 3DMolNet: A Generative Network for Molecular Structures
• Controllable Data Generation by Deep Learning: A Review
• Improving black-box optimization in VAE latent space using decoder uncertainty
• Generative Causal Explanations for Graph Neural Networks
• Permutation Invariant Graph Generation via Score-Based Generative Modeling
• A Model to Search for Synthesizable Molecules
• Reinforcement Learning for Molecular Design Guided by Quantum Mechanics
• Labeled Graph Generative Adversarial Networks
• A generative model for molecule generation based on chemical reaction trees
• Pointer Graph Networks
• Graph Generation with Variational Recurrent Neural Network
• Uncertainty quantification of molecular property prediction with Bayesian neural networks
• Goal-directed graph construction using reinforcement learning
• Graph Learning Network: A Structure Learning Algorithm
• Graph Coarsening with Neural Networks
• Model Extraction Attacks on Graph Neural Networks: Taxonomy and Realization
• Categorical Normalizing Flows via Continuous Transformations
• Variational Autoencoder for Anti-Cancer Drug Response Prediction
• Graph Deconvolutional Generation
• Multitask Learning on Graph Neural Networks: Learning Multiple Graph Centrality Measures with a Unified Network
• Generative Graph Convolutional Network for Growing Graphs
• Neural Graph Embedding Methods for Natural Language Processing
• Towards Interpretable Sparse Graph Representation Learning with Laplacian Pooling
• Disentangling Interpretable Generative Parameters of Random and Real-World Graphs
• Adversarial Learned Molecular Graph Inference and Generation
• Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning
• Black Box Recursive Translations for Molecular Optimization
• A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction
• DeepScaffold: a comprehensive tool for scaffold-based de novo drug discovery using deep learning
• Neural Architecture Optimization with Graph VAE
• Barking up the right tree: an approach to search over molecule synthesis DAGs
• Symmetry-Aware Actor-Critic for 3D Molecular Design
• Artificial Intelligence in Drug Discovery: Applications and Techniques
• Deep Generative Graph Distribution Learning for Synthetic Power Grids
• Graph Generators: State of the Art and Open Challenges
• Revisiting 2D Convolutional Neural Networks for Graph-based Applications
• Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery
• Constant Time Graph Neural Networks
• Neural Markov Logic Networks
• MEMO: A Deep Network for Flexible Combination of Episodic Memories
• GraphTune: A Learning-based Graph Generative Model with Tunable Structural Features
• Neural Turtle Graphics for Modeling City Road Layouts
• Non-Parametric Graph Learning for Bayesian Graph Neural Networks
• Local Propagation in Constraint-based Neural Network
• Spatial-Temporal Block and LSTM Network for Pedestrian Trajectories Prediction
• Chemical Structure Elucidation from Mass Spectrometry by Matching Substructures
• Building LEGO Using Deep Generative Models of Graphs
• DeepGG: a Deep Graph Generator
• Node Copying: A Random Graph Model for Effective Graph Sampling
• Deep Denerative Models for Drug Design and Response
• GraphTER: Unsupervised Learning of Graph Transformation Equivariant Representations via Auto-Encoding Node-wise Transformations
• Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation
• Signed Network Modeling Based on Structural Balance Theory
• Continuous Graph Flow
• Structure-Aware Generation Network for Recipe Generation from Images
• Discovering Dynamic Salient Regions for Spatio-Temporal Graph Neural Networks
• AMR Parsing with Action-Pointer Transformer
• Re-balancing Variational Autoencoder Loss for Molecule Sequence Generation
• Continuous-Depth Neural Models for Dynamic Graph Prediction
• The Infinity Mirror Test for Graph Models
• Goal directed molecule generation using Monte Carlo Tree Search
• DeepNC: Deep Generative Network Completion
• Discrete Object Generation with Reversible Inductive Construction
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• The Atlas for the Aspiring Network Scientist
• Physics-Constrained Predictive Molecular Latent Space Discovery with Graph Scattering Variational Autoencoder
• Auto-decoding Graphs
• SHADOWCAST: Controllable Graph Generation
• Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation
• Learning Knowledge Graph-based World Models of Textual Environments
• Generating a Doppelganger Graph: Resembling but Distinct
• Uncertainty quantification of molecular property prediction using Bayesian neural network models
• Sequence-guided protein structure determination using graph convolutional and recurrent networks
• Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learning
• Deep Generative Modeling in Network Science with Applications to Public Policy Research
• Fréchet ChemNet Distance: A metric for generative models for molecules in drug discovery
• A Graph Deep Learning Framework for High-Level Synthesis Design Space Exploration
• Scene Graph Modification Based on Natural Language Commands
• GeoGraph: Learning graph-based multi-view object detection with geometric cues end-to-end
• Partition and Code: learning how to compress graphs
• TGG: Transferable Graph Generation for Zero-shot and Few-shot Learning
• Personalized Hashtag Recommendation for Micro-videos
• DynGraph2Seq: Dynamic-Graph-to-Sequence Interpretable Learning for Health Stage Prediction in Online Health Forums
• On the Power of Edge Independent Graph Models
• Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration
• Towards explainable message passing networks for predicting carbon dioxide adsorption in metal-organic frameworks
• Neural Consciousness Flow
• Relational dynamic memory networks
• Amanuensis: The Programmer's Apprentice
• SGVAE: Sequential Graph Variational Autoencoder
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Learning to Represent and Predict Sets with Deep Neural Networks
• Sequential Graph Dependency Parser
• Semantic Graph Parsing with Recurrent Neural Network DAG Grammars
• Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining
• Adversarial Stein Training for Graph Energy Models
• GraphGen-Redux: a Fast and Lightweight Recurrent Model for labeled Graph Generation
• TD-GEN: Graph Generation With Tree Decomposition
• Generating the Graph Gestalt: Kernel-Regularized Graph Representation Learning
• Graph-Preserving Grid Layout: A Simple Graph Drawing Method for Graph Classification using CNNs
• Genetic Constrained Graph Variational Autoencoder for COVID-19 Drug Discovery