2.9k citations · 12k across the 144 of their papers we have counts for
23 papers · 1 filter
On Batch Normalisation for Approximate Bayesian Inference
Jishnu Mukhoti, Puneet K. Dokania, Philip H. S. Torr +1
We study batch normalisation in the context of variational inference methods in Bayesian neural networks, such as mean-field or MC Dropout. We show that batch-normalisation does no…
Semi-supervised Learning of Galaxy Morphology using Equivariant Transformer Variational Autoencoders
Mizu Nishikawa-Toomey, Lewis Smith, Yarin Gal
The growth in the number of galaxy images is much faster than the speed at which these galaxies can be labelled by humans. However, by leveraging the information present in the eve…
On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes
Tim G. J. Rudner, Oscar Key, Yarin Gal +1
We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) is…
A Bayesian Perspective on Training Speed and Model Selection
Clare Lyle, Lisa Schut, Binxin Ru +2
We take a Bayesian perspective to illustrate a connection between training speed and the marginal likelihood in linear models. This provides two major insights: first, that a measu…
Physics-informed GANs for Coastal Flood Visualization
Björn Lütjens, Brandon Leshchinskiy, Christian Requena-Mesa +8
As climate change increases the intensity of natural disasters, society needs better tools for adaptation. Floods, for example, are the most frequent natural disaster, but during h…
Interlocking Backpropagation: Improving depthwise model-parallelism
Aidan N. Gomez, Oscar Key, Kuba Perlin +4
The number of parameters in state of the art neural networks has drastically increased in recent years. This surge of interest in large scale neural networks has motivated the deve…
How Robust are the Estimated Effects of Nonpharmaceutical Interventions against COVID-19?
Mrinank Sharma, Sören Mindermann, Jan Markus Brauner +7
To what extent are effectiveness estimates of nonpharmaceutical interventions (NPIs) against COVID-19 influenced by the assumptions our models make? To answer this question, we inv…
Improving compute efficacy frontiers with SliceOut
Pascal Notin, Aidan N. Gomez, Joanna Yoo +1
Pushing forward the compute efficacy frontier in deep learning is critical for tasks that require frequent model re-training or workloads that entail training a large number of mod…
Single Shot Structured Pruning Before Training
Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar +2
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning befo…
Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models
Andrew Jesson, Sören Mindermann, Uri Shalit +1
Recommending the best course of action for an individual is a major application of individual-level causal effect estimation. This application is often needed in safety-critical do…
Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
Tim Z. Xiao, Aidan N. Gomez, Yarin Gal
We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure…
Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?
Angelos Filos, Panagiotis Tigas, Rowan McAllister +3
Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment, typically leading to arbitrary deductions and poorly-informed decisions. In pr…
Technology Readiness Levels for AI & ML
Alexander Lavin, Gregory Renard
The development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence…
Learning Invariant Representations for Reinforcement Learning without Reconstruction
Amy Zhang, Rowan McAllister, Roberto Calandra +2
We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstructio…
Speedy Performance Estimation for Neural Architecture Search
Binxin Ru, Clare Lyle, Lisa Schut +3
Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS). Traditional approaches fac…
Uncertainty Evaluation Metric for Brain Tumour Segmentation
Raghav Mehta, Angelos Filos, Yarin Gal +1
In this paper, we develop a metric designed to assess and rank uncertainty measures for the task of brain tumour sub-tissue segmentation in the BraTS 2019 sub-challenge on uncertai…
On the Benefits of Invariance in Neural Networks
Clare Lyle, Mark van der Wilk, Marta Kwiatkowska +2
Many real world data analysis problems exhibit invariant structure, and models that take advantage of this structure have shown impressive empirical performance, particularly in de…
Capsule Networks -- A Probabilistic Perspective
Lewis Smith, Lisa Schut, Yarin Gal +1
'Capsule' models try to explicitly represent the poses of objects, enforcing a linear relationship between an object's pose and that of its constituent parts. This modelling assump…
Baryons from Mesons: A Machine Learning Perspective
Yarin Gal, Vishnu Jejjala, Damian Kaloni Mayorga Pena +1
Quantum chromodynamics (QCD) is the theory of the strong interaction. The fundamental particles of QCD, quarks and gluons, carry colour charge and form colourless bound states at l…
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
Andreas Kirsch, Clare Lyle, Yarin Gal
The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models…
Invariant Causal Prediction for Block MDPs
Amy Zhang, Clare Lyle, Shagun Sodhani +5
Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. In this paper, we consider the problem o…
Uncertainty Estimation Using a Single Deep Deterministic Neural Network
Joost van Amersfoort, Lewis Smith, Yee Whye Teh +1
We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. Our approach, determin…
Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations
Sebastian Farquhar, Lewis Smith, Yarin Gal
We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive, and show this is not the c…