Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings

Abstract

In this paper, we propose an end-to-end graph learning framework, namely Iterative Deep Graph Learning (IDGL), for jointly and iteratively learning graph structure and graph embedding. The key rationale of IDGL is to learn a better graph structure based on better node embeddings, and vice versa (i.e., better node embeddings based on a better graph structure). Our iterative method dynamically stops when the learned graph structure approaches close enough to the graph optimized for the downstream prediction task. In addition, we cast the graph learning problem as a similarity metric learning problem and leverage adaptive graph regularization for controlling the quality of the learned graph. Finally, combining the anchor-based approximation technique, we further propose a scalable version of IDGL, namely IDGL-Anch, which significantly reduces the time and space complexity of IDGL without compromising the performance. Our extensive experiments on nine benchmarks show that our proposed IDGL models can consistently outperform or match the state-of-the-art baselines. Furthermore, IDGL can be more robust to adversarial graphs and cope with both transductive and inductive learning.

19 pages. Accepted by NeurIPS 2020. Final version

References in corpus (24)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Representation Learning on Graphs with Jumping Knowledge Networks
• Graph Transformer Networks
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• Learning Deep Generative Models of Graphs
• Robust Graph Learning from Noisy Data
• Adversarial Attack and Defense on Graph Data: A Survey
• DAGs with NO TEARS: Continuous Optimization for Structure Learning
• Learning Discrete Structures for Graph Neural Networks
• Efficient Graph Generation with Graph Recurrent Attention Networks
• Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks
• Learning Conditioned Graph Structures for Interpretable Visual Question Answering
• DAG-GNN: DAG Structure Learning with Graph Neural Networks
• Large Scale Graph Learning from Smooth Signals
• Graph Structure Learning for Robust Graph Neural Networks
• Variational Inference for Graph Convolutional Networks in the Absence of Graph Data and Adversarial Settings
• GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension
• Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model
• Graph Convolutional Networks with EigenPooling
• GRASPEL: Graph Spectral Learning at Scale

Cited by in corpus (24)

• Mining Latent Structures for Multimedia Recommendation
• A Comprehensive Survey on Deep Graph Representation Learning
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• Disentangled Contrastive Collaborative Filtering
• Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation
• Differentiable Graph Module (DGM) for Graph Convolutional Networks
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Deep Graph Matching and Searching for Semantic Code Retrieval
• Multilevel Graph Matching Networks for Deep Graph Similarity Learning
• Graph Neural Networks for Natural Language Processing: A Survey
• SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization
• Learning Strong Graph Neural Networks with Weak Information
• Position-aware Structure Learning for Graph Topology-imbalance by Relieving Under-reaching and Over-squashing
• Homophily-oriented Heterogeneous Graph Rewiring
• Learning Graph Neural Networks with Positive and Unlabeled Nodes
• Graph Condensation for Graph Neural Networks
• Sequential Recommendation with Graph Neural Networks
• Deep Contrastive Graph Learning with Clustering-Oriented Guidance
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks
• Adversarial Graph Disentanglement
• GCN-SL: Graph Convolutional Networks with Structure Learning for Graphs under Heterophily
• Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing
• Semantic Graph Neural Network with Multi-measure Learning for Semi-supervised Classification
• Multi-modal Graph Learning for Disease Prediction