2.9k citations · 12k across the 144 of their papers we have counts for
31 papers · 1 filter
Continual Learning via Sequential Function-Space Variational Inference
Tim G. J. Rudner, Freddie Bickford Smith, Qixuan Feng +2
Sequential Bayesian inference over predictive functions is a natural framework for continual learning from streams of data. However, applying it to neural networks has proved chall…
Tractable Function-Space Variational Inference in Bayesian Neural Networks
Tim G. J. Rudner, Zonghao Chen, Yee Whye Teh +1
Reliable predictive uncertainty estimation plays an important role in enabling the deployment of neural networks to safety-critical settings. A popular approach for estimating the…
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks
Neil Band, Tim G. J. Rudner, Qixuan Feng +6
Bayesian deep learning seeks to equip deep neural networks with the ability to precisely quantify their predictive uncertainty, and has promised to make deep learning more reliable…
Contrastive Representation Learning with Trainable Augmentation Channel
Masanori Koyama, Kentaro Minami, Takeru Miyato +1
In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations. However, depen…
Active Testing: Sample-Efficient Model Evaluation
Jannik Kossen, Sebastian Farquhar, Yarin Gal +1
We introduce a new framework for sample-efficient model evaluation that we call active testing. While approaches like active learning reduce the number of labels needed for model t…
On Statistical Bias In Active Learning: How and When To Fix It
Sebastian Farquhar, Yarin Gal, Tom Rainforth
Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution. We formalize…
Semi-supervised Learning of Galaxy Morphology using Equivariant Transformer Variational Autoencoders
Mizu Nishikawa-Toomey, Lewis Smith, Yarin Gal
The growth in the number of galaxy images is much faster than the speed at which these galaxies can be labelled by humans. However, by leveraging the information present in the eve…
On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes
Tim G. J. Rudner, Oscar Key, Yarin Gal +1
We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) is…
Speedy Performance Estimation for Neural Architecture Search
Binxin Ru, Clare Lyle, Lisa Schut +3
Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS). Traditional approaches fac…
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
Angelos Filos, Sebastian Farquhar, Aidan N. Gomez +6
Evaluation of Bayesian deep learning (BDL) methods is challenging. We often seek to evaluate the methods' robustness and scalability, assessing whether new tools give `better' unce…
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
Sebastian Farquhar, Michael Osborne, Yarin Gal
We propose Radial Bayesian Neural Networks (BNNs): a variational approximate posterior for BNNs which scales well to large models while maintaining a distribution over weight-space…
Differentially Private Continual Learning
Sebastian Farquhar, Yarin Gal
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient…
A Unifying Bayesian View of Continual Learning
Sebastian Farquhar, Yarin Gal
Some machine learning applications require continual learning - where data comes in a sequence of datasets, each is used for training and then permanently discarded. From a Bayesia…
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
Mohammad Emtiyaz Khan, Didrik Nielsen, Voot Tangkaratt +3
Uncertainty computation in deep learning is essential to design robust and reliable systems. Variational inference (VI) is a promising approach for such computation, but requires m…
Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
Yarin Gal, Lewis Smith
We prove, under two sufficient conditions, that idealised models can have no adversarial examples. We discuss which idealised models satisfy our conditions, and show that idealised…
Loss-Calibrated Approximate Inference in Bayesian Neural Networks
Adam D. Cobb, Stephen J. Roberts, Yarin Gal
Current approaches in approximate inference for Bayesian neural networks minimise the Kullback-Leibler divergence to approximate the true posterior over the weights. However, this…
Towards Robust Evaluations of Continual Learning
Sebastian Farquhar, Yarin Gal
Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually. Instead of assessing performance on challenging and…
Understanding Measures of Uncertainty for Adversarial Example Detection
Lewis Smith, Yarin Gal
Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence. But many meas…
BRUNO: A Deep Recurrent Model for Exchangeable Data
Iryna Korshunova, Jonas Degrave, Ferenc Huszár +3
We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations. Our model is provab…
Vprop: Variational Inference using RMSprop
Mohammad Emtiyaz Khan, Zuozhu Liu, Voot Tangkaratt +1
Many computationally-efficient methods for Bayesian deep learning rely on continuous optimization algorithms, but the implementation of these methods requires significant changes t…
Real Time Image Saliency for Black Box Classifiers
Piotr Dabkowski, Yarin Gal
In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier. We train a masking model to manipulate the scores of the classi…
Concrete Dropout
Yarin Gal, Jiri Hron, Alex Kendall
Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimate…
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
Yarin Gal, Zoubin Ghahramani
Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dr…
Dirichlet Fragmentation Processes
Hong Ge, Yarin Gal, Zoubin Ghahramani
Tree structures are ubiquitous in data across many domains, and many datasets are naturally modelled by unobserved tree structures. In this paper, first we review the theory of ran…