Bayesian Graph Neural Networks with Adaptive Connection Sampling

Abstract

We propose a unified framework for adaptive connection sampling in graph neural networks (GNNs) that generalizes existing stochastic regularization methods for training GNNs. The proposed framework not only alleviates over-smoothing and over-fitting tendencies of deep GNNs, but also enables learning with uncertainty in graph analytic tasks with GNNs. Instead of using fixed sampling rates or hand-tuning them as model hyperparameters in existing stochastic regularization methods, our adaptive connection sampling can be trained jointly with GNN model parameters in both global and local fashions. GNN training with adaptive connection sampling is shown to be mathematically equivalent to an efficient approximation of training Bayesian GNNs. Experimental results with ablation studies on benchmark datasets validate that adaptively learning the sampling rate given graph training data is the key to boost the performance of GNNs in semi-supervised node classification, less prone to over-smoothing and over-fitting with more robust prediction.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Improving neural networks by preventing co-adaptation of feature detectors
• Inductive Representation Learning on Large Graphs
• Categorical Reparameterization with Gumbel-Softmax
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Variational Graph Auto-Encoders
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Variational Dropout and the Local Reparameterization Trick
• Signal Recovery on Graphs: Variation Minimization
• Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
• Variational Graph Recurrent Neural Networks
• Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data
• Learnable Bernoulli Dropout for Bayesian Deep Learning
• Pairwise Supervised Hashing with Bernoulli Variational Auto-Encoder and Self-Control Gradient Estimator

Cited by in corpus (8)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Retention Time Prediction for Chromatographic Enantioseparation by Quantile Geometry-enhanced Graph Neural Network
• Bi-GCN: Binary Graph Convolutional Network
• Probabilistic Deep Learning to Quantify Uncertainty in Air Quality Forecasting
• Deep Graph Neural Networks with Shallow Subgraph Samplers
• Sampling methods for efficient training of graph convolutional networks: A survey
• BayReL: Bayesian Relational Learning for Multi-omics Data Integration
• LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs