Probabilistic Deep Learning with Probabilistic Neural Networks and Deep Probabilistic Models

Abstract

Probabilistic deep learning is deep learning that accounts for uncertainty, both model uncertainty and data uncertainty. It is based on the use of probabilistic models and deep neural networks. We distinguish two approaches to probabilistic deep learning: probabilistic neural networks and deep probabilistic models. The former employs deep neural networks that utilize probabilistic layers which can represent and process uncertainty; the latter uses probabilistic models that incorporate deep neural network components which capture complex non-linear stochastic relationships between the random variables. We discuss some major examples of each approach including Bayesian neural networks and mixture density networks (for probabilistic neural networks), and variational autoencoders, deep Gaussian processes and deep mixed effects models (for deep probabilistic models). TensorFlow Probability is a library for probabilistic modeling and inference which can be used for both approaches of probabilistic deep learning. We include its code examples for illustration.

arXiv commentarXiv admin note: text overlap with arXiv:1811.06622

References in corpus (5)

• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Bayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax
• Predictive Uncertainty Quantification with Compound Density Networks
• Joint Distributions for TensorFlow Probability
• Concept-Oriented Deep Learning: Generative Concept Representations

Cited by in corpus (2)

• Bayesian Neural Networks: Essentials
• Hybrid Bayesian Neural Networks with Functional Probabilistic Layers