Regression Prior Networks

Abstract

Prior Networks are a recently developed class of models which yield interpretable measures of uncertainty and have been shown to outperform state-of-the-art ensemble approaches on a range of tasks. They can also be used to distill an ensemble of models via Ensemble Distribution Distillation (EnD${}^2$), such that its accuracy, calibration and uncertainty estimates are retained within a single model. However, Prior Networks have so far been developed only for classification tasks. This work extends Prior Networks and EnD${}^2$ to regression tasks by considering the Normal-Wishart distribution. The properties of Regression Prior Networks are demonstrated on synthetic data, selected UCI datasets and a monocular depth estimation task, where they yield performance competitive with ensemble approaches.

References in corpus (14)

• Efficient Estimation of Word Representations in Vector Space
• Distilling the Knowledge in a Neural Network
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Evidential Deep Learning to Quantify Classification Uncertainty
• Deep Ensembles: A Loss Landscape Perspective
• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
• High Quality Monocular Depth Estimation via Transfer Learning
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning
• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning
• Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness
• Hydra: Preserving Ensemble Diversity for Model Distillation

Cited by in corpus (4)

• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
• Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
• NOMU: Neural Optimization-based Model Uncertainty