Convolutional Normalizing Flows for Deep Gaussian Processes

Abstract

Deep Gaussian processes (DGPs), a hierarchical composition of GP models, have successfully boosted the expressive power of their single-layer counterpart. However, it is impossible to perform exact inference in DGPs, which has motivated the recent development of variational inference-based methods. Unfortunately, either these methods yield a biased posterior belief or it is difficult to evaluate their convergence. This paper introduces a new approach for specifying flexible, arbitrarily complex, and scalable approximate posterior distributions. The posterior distribution is constructed through a normalizing flow (NF) which transforms a simple initial probability into a more complex one through a sequence of invertible transformations. Moreover, a novel convolutional normalizing flow (CNF) is developed to improve the time efficiency and capture dependency between layers. Empirical evaluation shows that CNF DGP outperforms the state-of-the-art approximation methods for DGPs.

arXiv commentTo appear in Proceedings of the International Joint Conference on Neural Networks 2021 (IJCNN'21). arXiv admin note: text overlap with arXiv:1910.11998

References in corpus (17)

• Auto-Encoding Variational Bayes
• Variational Inference with Normalizing Flows
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• Random Feature Expansions for Deep Gaussian Processes
• Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
• Decentralized Data Fusion and Active Sensing with Mobile Sensors for Modeling and Predicting Spatiotemporal Traffic Phenomena
• Nested Variational Compression in Deep Gaussian Processes
• Active Markov Information-Theoretic Path Planning for Robotic Environmental Sensing
• Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations
• Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds
• Implicit Posterior Variational Inference for Deep Gaussian Processes
• R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games
• Bayesian Optimization with Binary Auxiliary Information
• Private Outsourced Bayesian Optimization
• Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression
• An Information-Theoretic Framework for Unifying Active Learning Problems