URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks

Abstract

While deep learning methods continue to improve in predictive accuracy on a wide range of application domains, significant issues remain with other aspects of their performance including their ability to quantify uncertainty and their robustness. Recent advances in approximate Bayesian inference hold significant promise for addressing these concerns, but the computational scalability of these methods can be problematic when applied to large-scale models. In this paper, we describe initial work on the development ofURSABench(the Uncertainty, Robustness, Scalability, and Accu-racy Benchmark), an open-source suite of bench-marking tools for comprehensive assessment of approximate Bayesian inference methods with a focus on deep learning-based classification tasks

Presented at the ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning

References in corpus (6)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Expectation Propagation for approximate Bayesian inference
• Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
• Subspace Inference for Bayesian Deep Learning
• Introducing an Explicit Symplectic Integration Scheme for Riemannian Manifold Hamiltonian Monte Carlo
• Assessing the Adversarial Robustness of Monte Carlo and Distillation Methods for Deep Bayesian Neural Network Classification