Uncertainty in the Variational Information Bottleneck

Abstract

We present a simple case study, demonstrating that Variational Information Bottleneck (VIB) can improve a network's classification calibration as well as its ability to detect out-of-distribution data. Without explicitly being designed to do so, VIB gives two natural metrics for handling and quantifying uncertainty.

arXiv comment10 pages, 7 figures. Accepted to UAI 2018 - Uncertainty in Deep Learning Workshop

References in corpus (4)

• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Deep Variational Information Bottleneck

Cited by in corpus (37)

• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• TabTransformer: Tabular Data Modeling Using Contextual Embeddings
• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
• Nonlinear Information Bottleneck
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Input complexity and out-of-distribution detection with likelihood-based generative models
• The Conditional Entropy Bottleneck
• Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
• Modeling Uncertainty with Hedged Instance Embedding
• Probabilistic Autoencoder
• Caveats for information bottleneck in deterministic scenarios
• Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
• InfoAT: Improving Adversarial Training Using the Information Bottleneck Principle
• Density of States Estimation for Out-of-Distribution Detection
• Preventing Posterior Collapse with delta-VAEs
• The Hidden Uncertainty in a Neural Networks Activations
• Predictive Uncertainty Quantification with Compound Density Networks
• Further Analysis of Outlier Detection with Deep Generative Models
• Hierarchical VAEs Know What They Don't Know
• Likelihood Ratios for Out-of-Distribution Detection
• Mutual Information Gradient Estimation for Representation Learning
• Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
• Bounds all around: training energy-based models with bidirectional bounds
• Dueling Decoders: Regularizing Variational Autoencoder Latent Spaces
• An Algorithm for Out-Of-Distribution Attack to Neural Network Encoder
• On the Out-of-distribution Generalization of Probabilistic Image Modelling
• A Modular Deep Learning Pipeline for Galaxy-Scale Strong Gravitational Lens Detection and Modeling
• Flexible Variational Information Bottleneck: Achieving Diverse Compression with a Single Training
• Rectangular Flows for Manifold Learning
• DOI: Divergence-based Out-of-Distribution Indicators via Deep Generative Models
• Unsupervised Class-Incremental Learning Through Confusion
• Revisiting Explicit Regularization in Neural Networks for Well-Calibrated Predictive Uncertainty
• Self-Paced Uncertainty Estimation for One-shot Person Re-Identification
• Reliable Estimation of Individual Treatment Effect with Causal Information Bottleneck
• Information Theoretic Structured Generative Modeling