A Survey on Bayesian Deep Learning

Abstract

A comprehensive artificial intelligence system needs to not only perceive the environment with different `senses' (e.g., seeing and hearing) but also infer the world's conditional (or even causal) relations and corresponding uncertainty. The past decade has seen major advances in many perception tasks such as visual object recognition and speech recognition using deep learning models. For higher-level inference, however, probabilistic graphical models with their Bayesian nature are still more powerful and flexible. In recent years, Bayesian deep learning has emerged as a unified probabilistic framework to tightly integrate deep learning and Bayesian models. In this general framework, the perception of text or images using deep learning can boost the performance of higher-level inference and in turn, the feedback from the inference process is able to enhance the perception of text or images. This survey provides a comprehensive introduction to Bayesian deep learning and reviews its recent applications on recommender systems, topic models, control, etc. Besides, we also discuss the relationship and differences between Bayesian deep learning and other related topics such as Bayesian treatment of neural networks. For a constantly updating project page, please refer to https://github.com/js05212/BayesianDeepLearning-Survey.

Published in ACM Computing Surveys (CSUR) 2020. Constantly updating project page at https://github.com/js05212/BayesianDeepLearning-Survey

References in corpus (10)

• Sequence to Sequence Learning with Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• A Recurrent Latent Variable Model for Sequential Data
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Marginalized Denoising Autoencoders for Domain Adaptation
• Bayesian Dark Knowledge
• Collaborative Topic Regression with Social Matrix Factorization for Recommendation Systems
• Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks
• Natural-Parameter Networks: A Class of Probabilistic Neural Networks
• Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC

Cited by in corpus (7)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Remote patient monitoring using artificial intelligence: Current state, applications, and challenges
• The Relevance of Bayesian Layer Positioning to Model Uncertainty in Deep Bayesian Active Learning
• Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
• Mitigating Uncertainty in Document Classification
• Sampling-Free Learning of Bayesian Quantized Neural Networks
• Learning Probabilistic Programs Using Backpropagation