A Comprehensive Survey on Deep Graph Representation Learning

Abstract

Graph representation learning aims to effectively encode high-dimensional sparse graph-structured data into low-dimensional dense vectors, which is a fundamental task that has been widely studied in a range of fields, including machine learning and data mining. Classic graph embedding methods follow the basic idea that the embedding vectors of interconnected nodes in the graph can still maintain a relatively close distance, thereby preserving the structural information between the nodes in the graph. However, this is sub-optimal due to: (i) traditional methods have limited model capacity which limits the learning performance; (ii) existing techniques typically rely on unsupervised learning strategies and fail to couple with the latest learning paradigms; (iii) representation learning and downstream tasks are dependent on each other which should be jointly enhanced. With the remarkable success of deep learning, deep graph representation learning has shown great potential and advantages over shallow (traditional) methods, there exist a large number of deep graph representation learning techniques have been proposed in the past decade, especially graph neural networks. In this survey, we conduct a comprehensive survey on current deep graph representation learning algorithms by proposing a new taxonomy of existing state-of-the-art literature. Specifically, we systematically summarize the essential components of graph representation learning and categorize existing approaches by the ways of graph neural network architectures and the most recent advanced learning paradigms. Moreover, this survey also provides the practical and promising applications of deep graph representation learning. Last but not least, we state new perspectives and suggest challenging directions which deserve further investigations in the future.

arXiv commentAccepted by Neural Networks 2024

References in corpus (153)

• Efficient Estimation of Word Representations in Vector Space
• Auto-Encoding Variational Bayes
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• A Comprehensive Survey on Graph Neural Networks
• DeepWalk: Online Learning of Social Representations
• Denoising Diffusion Probabilistic Models
• LINE: Large-scale Information Network Embedding
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Bootstrap your own latent: A new approach to self-supervised Learning
• Automatic chemical design using a data-driven continuous representation of molecules
• Neural Graph Collaborative Filtering
• Spectral Networks and Locally Connected Networks on Graphs
• SchNet - a deep learning architecture for molecules and materials
• European Union regulations on algorithmic decision-making and a "right to explanation"
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Graph Embedding Techniques, Applications, and Performance: A Survey
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Discrete Signal Processing on Graphs
• Session-based Recommendation with Graph Neural Networks
• Temporal Ensembling for Semi-Supervised Learning
• Deep Convolutional Networks on Graph-Structured Data
• Graph Neural Network for Traffic Forecasting: A Survey
• Graph Convolutional Matrix Completion
• Graph Contrastive Learning with Adaptive Augmentation
• Generative Modeling by Estimating Gradients of the Data Distribution
• Molecular Contrastive Learning of Representations via Graph Neural Networks
• Hierarchical Graph Representation Learning with Differentiable Pooling
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks
• Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting
• Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting
• Disentangled Graph Collaborative Filtering
• Global Context Enhanced Graph Neural Networks for Session-based Recommendation
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Graph Learning: A Survey
• Hypergraph Contrastive Collaborative Filtering
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank
• Deep Graph Contrastive Representation Learning
• A Survey on Graph Kernels
• GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
• Rethinking the Item Order in Session-based Recommendation with Graph Neural Networks
• Constrained Graph Variational Autoencoders for Molecule Design
• A Generalization of Transformer Networks to Graphs
• A Gentle Introduction to Deep Learning for Graphs
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation
• Equivariant message passing for the prediction of tensorial properties and molecular spectra
• DeeperGCN: All You Need to Train Deeper GCNs
• OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features
• Principal Neighbourhood Aggregation for Graph Nets
• The Frontiers of Fairness in Machine Learning
• Graph Representation Learning: A Survey
• MoFlow: An Invertible Flow Model for Generating Molecular Graphs
• Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings
• User-oriented Fairness in Recommendation
• GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation
• Inverse design of 3d molecular structures with conditional generative neural networks
• Learning from Protein Structure with Geometric Vector Perceptrons
• Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks
• Transferring Robustness for Graph Neural Network Against Poisoning Attacks
• Learning to design drug-like molecules in three-dimensional space using deep generative models
• Diffusion Improves Graph Learning
• Knowledge Graph Self-Supervised Rationalization for Recommendation
• Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation
• Graph-Bert: Only Attention is Needed for Learning Graph Representations
• Few-Shot Graph Learning for Molecular Property Prediction
• BotRGCN: Twitter Bot Detection with Relational Graph Convolutional Networks
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Molecular geometry prediction using a deep generative graph neural network
• EDITS: Modeling and Mitigating Data Bias for Graph Neural Networks
• Representing Long-Range Context for Graph Neural Networks with Global Attention
• Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels
• Automated Self-Supervised Learning for Recommendation
• One-Class Graph Neural Networks for Anomaly Detection in Attributed Networks
• GNNGuard: Defending Graph Neural Networks against Adversarial Attacks
• Hierarchical Graph Pooling with Structure Learning
• e3nn: Euclidean Neural Networks
• ASGN: An Active Semi-supervised Graph Neural Network for Molecular Property Prediction
• Self-supervised Learning on Graphs: Deep Insights and New Direction
• Molecule Attention Transformer
• GAG: Global Attributed Graph Neural Network for Streaming Session-based Recommendation
• Structure-based Drug Design with Equivariant Diffusion Models
• GraphNVP: An Invertible Flow Model for Generating Molecular Graphs
• DisenPOI: Disentangling Sequential and Geographical Influence for Point-of-Interest Recommendation
• 3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction
• GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text
• Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
• Learning Graph-Level Representation for Drug Discovery
• Deeply learning molecular structure-property relationships using attention- and gate-augmented graph convolutional network
• Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets
• DeepRobust: A PyTorch Library for Adversarial Attacks and Defenses
• Collaboration-Aware Graph Convolutional Network for Recommender Systems
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
• Learning Graph ODE for Continuous-Time Sequential Recommendation
• A Survey of Adversarial Learning on Graphs
• DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
• GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction
• Dual Space Graph Contrastive Learning
• DEFactor: Differentiable Edge Factorization-based Probabilistic Graph Generation
• Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification
• Batch Virtual Adversarial Training for Graph Convolutional Networks
• Towards Semi-supervised Universal Graph Classification
• Generating 3D Molecules for Target Protein Binding
• Learning Gradient Fields for Molecular Conformation Generation
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language
• Semi-Supervised Hierarchical Graph Classification
• RetGK: Graph Kernels based on Return Probabilities of Random Walks
• Property-Aware Relation Networks for Few-Shot Molecular Property Prediction
• Multi-View Graph Neural Networks for Molecular Property Prediction
• Multi-hop Attention Graph Neural Network
• LieTransformer: Equivariant self-attention for Lie Groups
• Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks
• Geometrically Equivariant Graph Neural Networks: A Survey
• DiffBP: Generative Diffusion of 3D Molecules for Target Protein Binding
• Nested Graph Neural Networks
• KGNN: Harnessing Kernel-based Networks for Semi-supervised Graph Classification
• Subgroup Generalization and Fairness of Graph Neural Networks
• MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design
• A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective
• Out-Of-Distribution Generalization on Graphs: A Survey
• 3DMolNet: A Generative Network for Molecular Structures
• CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning
• Direct Molecular Conformation Generation
• Object Hider: Adversarial Patch Attack Against Object Detectors
• Convolutional Kernel Networks for Graph-Structured Data
• Deriving Neural Architectures from Sequence and Graph Kernels
• A Causal Lens for Controllable Text Generation
• Graph Inductive Biases in Transformers without Message Passing
• Deep Multi-View Spatiotemporal Virtual Graph Neural Network for Significant Citywide Ride-hailing Demand Prediction
• Rethinking the Expressive Power of GNNs via Graph Biconnectivity
• Graph Learning Network: A Structure Learning Algorithm
• RPT: Toward Transferable Model on Heterogeneous Researcher Data via Pre-Training
• Graph Neural Networks in Intelligent Transportation Systems: Advances, Applications and Trends
• Graph Anomaly Detection with Unsupervised GNNs
• Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
• Label-invariant Augmentation for Semi-Supervised Graph Classification
• FIMO: A Challenge Formal Dataset for Automated Theorem Proving
• Zero-Shot 3D Drug Design by Sketching and Generating
• Retrieval-based Controllable Molecule Generation
• Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference
• MSGNN: A Spectral Graph Neural Network Based on a Novel Magnetic Signed Laplacian
• Proof of the Theory-to-Practice Gap in Deep Learning via Sampling Complexity bounds for Neural Network Approximation Spaces
• Impact-Oriented Contextual Scholar Profiling using Self-Citation Graphs
• MxPool: Multiplex Pooling for Hierarchical Graph Representation Learning
• Are More Layers Beneficial to Graph Transformers?
• GraphVF: Controllable Protein-Specific 3D Molecule Generation with Variational Flow
• Graph Neural Tangent Kernel: Convergence on Large Graphs
• PGODE: Towards High-quality System Dynamics Modeling
• Graph Self-Supervised Learning for Endoscopic Image Matching
• Geometry-Complete Perceptron Networks for 3D Molecular Graphs

Cited by in corpus (9)

• Learning Graph ODE for Continuous-Time Sequential Recommendation
• GALA: Graph Diffusion-based Alignment with Jigsaw for Source-free Domain Adaptation
• Disambiguated Node Classification with Graph Neural Networks
• Negative-Free Self-Supervised Gaussian Embedding of Graphs
• SPARK: Adaptive Low-Rank Knowledge Graph Modeling in Hybrid Geometric Spaces for Recommendation
• An Efficient Hybridization of Graph Representation Learning and Metaheuristics for the Constrained Incremental Graph Drawing Problem
• Structure-preserving contrastive learning for spatial time series
• Generic Multimodal Spatially Graph Network for Spatially Embedded Network Representation Learning
• Rethinking Graph Domain Adaptation: A Spectral Contrastive Perspective