Bayesian graph convolutional neural networks via tempered MCMC

Abstract

Deep learning models, such as convolutional neural networks, have long been applied to image and multi-media tasks, particularly those with structured data. More recently, there has been more attention to unstructured data that can be represented via graphs. These types of data are often found in health and medicine, social networks, and research data repositories. Graph convolutional neural networks have recently gained attention in the field of deep learning that takes advantage of graph-based data representation with automatic feature extraction via convolutions. Given the popularity of these methods in a wide range of applications, robust uncertainty quantification is vital. This remains a challenge for large models and unstructured datasets. Bayesian inference provides a principled approach to uncertainty quantification of model parameters for deep learning models. Although Bayesian inference has been used extensively elsewhere, its application to deep learning remains limited due to the computational requirements of the Markov Chain Monte Carlo (MCMC) methods. Recent advances in parallel computing and advanced proposal schemes in MCMC sampling methods has opened the path for Bayesian deep learning. In this paper, we present Bayesian graph convolutional neural networks that employ tempered MCMC sampling with Langevin-gradient proposal distribution implemented via parallel computing. Our results show that the proposed method can provide accuracy similar to advanced optimisers while providing uncertainty quantification for key benchmark problems.

References in corpus (26)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Neural Networks: An Overview
• A Comprehensive Survey on Graph Neural Networks
• Neural Message Passing for Quantum Chemistry
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Exchange Monte Carlo Method and Application to Spin Glass Simulations
• Graph Neural Networks: A Review of Methods and Applications
• Simplifying Graph Convolutional Networks
• Parallel Tempering Algorithm for Conformational Studies of Biological Molecules
• Representation Learning on Graphs with Jumping Knowledge Networks
• Variational Graph Auto-Encoders
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• A Conceptual Introduction to Hamiltonian Monte Carlo
• Diffusion-Convolutional Neural Networks
• Large-Scale Learnable Graph Convolutional Networks
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Markov Chain Monte Carlo: Can We Trust the Third Significant Figure?
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• Stochastic Training of Graph Convolutional Networks with Variance Reduction
• GraphGAN: Graph Representation Learning with Generative Adversarial Nets
• Graph Neural Networks in Particle Physics
• GMNN: Graph Markov Neural Networks
• Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
• Deep Learning: A Bayesian Perspective
• Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting
• Multi-core parallel tempering Bayeslands for basin and landscape evolution

Cited by in corpus (5)

• SMOTified-GAN for class imbalanced pattern classification problems
• COVID-19 sentiment analysis via deep learning during the rise of novel cases
• A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications
• Bayesian neural networks via MCMC: a Python-based tutorial
• Revisiting Bayesian Autoencoders with MCMC