Predictive Uncertainty through Quantization
arXiv:1810.05500
Abstract
High-risk domains require reliable confidence estimates from predictive models. Deep latent variable models provide these, but suffer from the rigid variational distributions used for tractable inference, which err on the side of overconfidence. We propose Stochastic Quantized Activation Distributions (SQUAD), which imposes a flexible yet tractable distribution over discretized latent variables. The proposed method is scalable, self-normalizing and sample efficient. We demonstrate that the model fully utilizes the flexible distribution, learns interesting non-linearities, and provides predictive uncertainty of competitive quality.
References in corpus (7)
- Auto-Encoding Variational Bayes
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- The information bottleneck method
- Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
- Deep Variational Information Bottleneck
- Distributional Reinforcement Learning with Quantile Regression
- Batch-normalized Maxout Network in Network