Deep Sub-Ensembles for Fast Uncertainty Estimation in Image Classification
arXiv:1910.08168
Abstract
Fast estimates of model uncertainty are required for many robust robotics applications. Deep Ensembles provides state of the art uncertainty without requiring Bayesian methods, but still it is computationally expensive. In this paper we propose deep sub-ensembles, an approximation to deep ensembles where the core idea is to ensemble only the layers close to the output, and not the whole model. With ResNet-20 on the CIFAR10 dataset, we obtain 1.5-2.5 speedup over a Deep Ensemble, with a small increase in error and NLL, and similarly up to 5-15 speedup with a VGG-like network on the SVHN dataset. Our results show that this idea enables a trade-off between error and uncertainty quality versus computational performance.
7 pages, 8 figures, Bayesian Deep Learning Workshop 2019 @ NeurIPS 2019, camera ready
References in corpus (1)
Cited by in corpus (3)
- Non-Intrusive Reduced-Order Modeling Using Uncertainty-Aware Deep Neural Networks and Proper Orthogonal Decomposition: Application to Flood Modeling
- Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentation
- Hey Human, If your Facial Emotions are Uncertain, You Should Use Bayesian Neural Networks!