495 citations · 1.1k across the 25 of their papers we have counts for
25 papers
Do Multilingual LLMs Think In English?
Lisa Schut, Yarin Gal, Sebastian Farquhar
Large language models (LLMs) have multilingual capabilities and can solve tasks across various languages. However, we show that current LLMs make key decisions in a representation…
Model evaluation for extreme risks
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel +18
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to ca…
Prediction-Oriented Bayesian Active Learning
Freddie Bickford Smith, Andreas Kirsch, Sebastian Farquhar +3
Information-theoretic approaches to active learning have traditionally focused on maximising the information gathered about the model parameters, most commonly by optimising the BA…
Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
Lorenz Kuhn, Yarin Gal, Sebastian Farquhar
We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of…
CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models
Lorenz Kuhn, Yarin Gal, Sebastian Farquhar
Users often ask dialogue systems ambiguous questions that require clarification. We show that current language models rarely ask users to clarify ambiguous questions and instead pr…
Understanding Approximation for Bayesian Inference in Neural Networks
Sebastian Farquhar
Bayesian inference has theoretical attractions as a principled framework for reasoning about beliefs. However, the motivations of Bayesian inference which claim it to be the only '…
Do Bayesian Neural Networks Need To Be Fully Stochastic?
Mrinank Sharma, Sebastian Farquhar, Eric Nalisnick +1
We investigate the benefit of treating all the parameters in a Bayesian neural network stochastically and find compelling theoretical and empirical evidence that this standard cons…
Discovering Agents
Zachary Kenton, Ramana Kumar, Sebastian Farquhar +3
Causal models of agents have been used to analyse the safety aspects of machine learning systems. But identifying agents is non-trivial -- often the causal model is just assumed by…
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt
Sören Mindermann, Jan Brauner, Muhammed Razzak +8
Training on web-scale data can take months. But most computation and time is wasted on redundant and noisy points that are already learnt or not learnable. To accelerate training,…
Path-Specific Objectives for Safer Agent Incentives
Sebastian Farquhar, Ryan Carey, Tom Everitt
We present a general framework for training safe agents whose naive incentives are unsafe. As an example, manipulative or deceptive behaviour can improve rewards but should be avoi…
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients
Milad Alizadeh, Shyam A. Tailor, Luisa M Zintgraf +4
Pruning neural networks at initialization would enable us to find sparse models that retain the accuracy of the original network while consuming fewer computational resources for t…
Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
Jannik Kossen, Sebastian Farquhar, Yarin Gal +1
We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels ar…
Prioritized training on points that are learnable, worth learning, and not yet learned (workshop version)
Sören Mindermann, Muhammed Razzak, Winnie Xu +7
We introduce Goldilocks Selection, a technique for faster model training which selects a sequence of training points that are "just right". We propose an information-theoretic acqu…
Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning
Andreas Kirsch, Sebastian Farquhar, Parmida Atighehchian +3
We examine a simple stochastic strategy for adapting well-known single-point acquisition functions to allow batch active learning. Unlike acquiring the top-K points from the pool s…
Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
Zachary Nado, Neil Band, Mark Collier +23
High-quality estimates of uncertainty and robustness are crucial for numerous real-world applications, especially for deep learning which underlies many deployed ML systems. The ab…
Active Testing: Sample-Efficient Model Evaluation
Jannik Kossen, Sebastian Farquhar, Yarin Gal +1
We introduce a new framework for sample-efficient model evaluation that we call active testing. While approaches like active learning reduce the number of labels needed for model t…
On Statistical Bias In Active Learning: How and When To Fix It
Sebastian Farquhar, Yarin Gal, Tom Rainforth
Active learning is a powerful tool when labelling data is expensive, but it introduces a bias because the training data no longer follows the population distribution. We formalize…
Single Shot Structured Pruning Before Training
Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar +2
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning befo…
Liberty or Depth: Deep Bayesian Neural Nets Do Not Need Complex Weight Posterior Approximations
Sebastian Farquhar, Lewis Smith, Yarin Gal
We challenge the longstanding assumption that the mean-field approximation for variational inference in Bayesian neural networks is severely restrictive, and show this is not the c…
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
Angelos Filos, Sebastian Farquhar, Aidan N. Gomez +6
Evaluation of Bayesian deep learning (BDL) methods is challenging. We often seek to evaluate the methods' robustness and scalability, assessing whether new tools give `better' unce…
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
Sebastian Farquhar, Michael Osborne, Yarin Gal
We propose Radial Bayesian Neural Networks (BNNs): a variational approximate posterior for BNNs which scales well to large models while maintaining a distribution over weight-space…
Differentially Private Continual Learning
Sebastian Farquhar, Yarin Gal
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient…
A Unifying Bayesian View of Continual Learning
Sebastian Farquhar, Yarin Gal
Some machine learning applications require continual learning - where data comes in a sequence of datasets, each is used for training and then permanently discarded. From a Bayesia…
Towards Robust Evaluations of Continual Learning
Sebastian Farquhar, Yarin Gal
Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually. Instead of assessing performance on challenging and…