Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs

Abstract

Uncertainty quantification is crucial for building reliable and trustable machine learning systems. We propose to estimate uncertainty in recurrent neural networks (RNNs) via stochastic discrete state transitions over recurrent timesteps. The uncertainty of the model can be quantified by running a prediction several times, each time sampling from the recurrent state transition distribution, leading to potentially different results if the model is uncertain. Alongside uncertainty quantification, our proposed method offers several advantages in different settings. The proposed method can (1) learn deterministic and probabilistic automata from data, (2) learn well-calibrated models on real-world classification tasks, (3) improve the performance of out-of-distribution detection, and (4) control the exploration-exploitation trade-off in reinforcement learning.

References in corpus (13)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Improving neural networks by preventing co-adaptation of feature detectors
• Categorical Reparameterization with Gumbel-Softmax
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Learning Latent Dynamics for Planning from Pixels
• Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces
• Learning Deterministic Weighted Automata with Queries and Counterexamples
• Calibrated Model-Based Deep Reinforcement Learning