On Efficient Uncertainty Estimation for Resource-Constrained Mobile Applications

Abstract

Deep neural networks have shown great success in prediction quality while reliable and robust uncertainty estimation remains a challenge. Predictive uncertainty supplements model predictions and enables improved functionality of downstream tasks including embedded and mobile applications, such as virtual reality, augmented reality, sensor fusion, and perception. These applications often require a compromise in complexity to obtain uncertainty estimates due to very limited memory and compute resources. We tackle this problem by building upon Monte Carlo Dropout (MCDO) models using the Axolotl framework; specifically, we diversify sampled subnetworks, leverage dropout patterns, and use a branching technique to improve predictive performance while maintaining fast computations. We conduct experiments on (1) a multi-class classification task using the CIFAR10 dataset, and (2) a more complex human body segmentation task. Our results show the effectiveness of our approach by reaching close to Deep Ensemble prediction quality and uncertainty estimation, while still achieving faster inference on resource-limited mobile platforms.

7 pages; Accepted at the Bayesian Deep Learning Workshop, NeurIPS 2021

References in corpus (7)

• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Deep Ensembles: A Loss Landscape Perspective
• Training independent subnetworks for robust prediction
• Stochastic-YOLO: Efficient Probabilistic Object Detection under Dataset Shifts
• URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks
• Improving compute efficacy frontiers with SliceOut