642 citations · 3.9k across the 143 of their papers we have counts for
63 papers · 1 filter
A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning
Minyoung Kim, Timothy M. Hospedales
We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning, including hyperparameter optimization, loss function learning, few-shot learn…
FedL2P: Federated Learning to Personalize
Royson Lee, Minyoung Kim, Da Li +4
Federated learning (FL) research has made progress in developing algorithms for distributed learning of global models, as well as algorithms for local personalization of those comm…
Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Months
Fady Rezk, Antreas Antoniou, Henry Gouk +1
We analyze VeLO (versatile learned optimizer), the largest scale attempt to train a general purpose "foundational" optimizer to date. VeLO was trained on thousands of machine learn…
BayesDLL: Bayesian Deep Learning Library
Minyoung Kim, Timothy Hospedales
We release a new Bayesian neural network library for PyTorch for large-scale deep networks. Our library implements mainstream approximate Bayesian inference algorithms: variational…
Better Practices for Domain Adaptation
Linus Ericsson, Da Li, Timothy M. Hospedales
Distribution shifts are all too common in real-world applications of machine learning. Domain adaptation (DA) aims to address this by providing various frameworks for adapting mode…
Evaluating the Evaluators: Are Current Few-Shot Learning Benchmarks Fit for Purpose?
Luísa Shimabucoro, Timothy Hospedales, Henry Gouk
Numerous benchmarks for Few-Shot Learning have been proposed in the last decade. However all of these benchmarks focus on performance averaged over many tasks, and the question of…
A Hierarchical Bayesian Model for Deep Few-Shot Meta Learning
Minyoung Kim, Timothy Hospedales
We propose a novel hierarchical Bayesian model for learning with a large (possibly infinite) number of tasks/episodes, which suits well the few-shot meta learning problem. We consi…
Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks
Martin Ferianc, Ondrej Bohdal, Timothy Hospedales +1
Enhancing the generalisation abilities of neural networks (NNs) through integrating noise such as MixUp or Dropout during training has emerged as a powerful and adaptable technique…
MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations
Calum Heggan, Tim Hospedales, Sam Budgett +1
Contrastive self-supervised learning has gained attention for its ability to create high-quality representations from large unlabelled data sets. A key reason that these powerful f…
FedHB: Hierarchical Bayesian Federated Learning
Minyoung Kim, Timothy Hospedales
We propose a novel hierarchical Bayesian approach to Federated Learning (FL), where our model reasonably describes the generative process of clients' local data via hierarchical Ba…
ChiroDiff: Modelling chirographic data with Diffusion Models
Ayan Das, Yongxin Yang, Timothy Hospedales +2
Generative modelling over continuous-time geometric constructs, a.k.a such as handwriting, sketches, drawings etc., have been accomplished through autoregressive distributions. Suc…
Self-Supervised Multimodal Learning: A Survey
Yongshuo Zong, Oisin Mac Aodha, Timothy Hospedales
Multimodal learning, which aims to understand and analyze information from multiple modalities, has achieved substantial progress in the supervised regime in recent years. However,…
Domain Generalisation via Domain Adaptation: An Adversarial Fourier Amplitude Approach
Minyoung Kim, Da Li, Timothy Hospedales
We tackle the domain generalisation (DG) problem by posing it as a domain adaptation (DA) task where we adversarially synthesise the worst-case target domain and adapt a model to t…
Federated Learning for Inference at Anytime and Anywhere
Zicheng Liu, Da Li, Javier Fernandez-Marques +6
Federated learning has been predominantly concerned with collaborative training of deep networks from scratch, and especially the many challenges that arise, such as communication…
MEDFAIR: Benchmarking Fairness for Medical Imaging
Yongshuo Zong, Yongxin Yang, Timothy Hospedales
A multitude of work has shown that machine learning-based medical diagnosis systems can be biased against certain subgroups of people. This has motivated a growing number of bias m…
Attacking Adversarial Defences by Smoothing the Loss Landscape
Panagiotis Eustratiadis, Henry Gouk, Da Li +1
This paper investigates a family of methods for defending against adversarial attacks that owe part of their success to creating a noisy, discontinuous, or otherwise rugged loss la…
HyperInvariances: Amortizing Invariance Learning
Ruchika Chavhan, Henry Gouk, Jan Stühmer +1
Providing invariances in a given learning task conveys a key inductive bias that can lead to sample-efficient learning and good generalisation, if correctly specified. However, the…
Feed-Forward Latent Domain Adaptation
Ondrej Bohdal, Da Li, Shell Xu Hu +1
We study a new highly-practical problem setting that enables resource-constrained edge devices to adapt a pre-trained model to their local data distributions. Recognizing that devi…
Fisher SAM: Information Geometry and Sharpness Aware Minimisation
Minyoung Kim, Da Li, Shell Xu Hu +1
Recent sharpness-aware minimisation (SAM) is known to find flat minima which is beneficial for better generalisation with improved robustness. SAM essentially modifies the loss fun…
Meta Mirror Descent: Optimiser Learning for Fast Convergence
Boyan Gao, Henry Gouk, Hae Beom Lee +1
Optimisers are an essential component for training machine learning models, and their design influences learning speed and generalisation. Several studies have attempted to learn m…
Gaussian Process Meta Few-shot Classifier Learning via Linear Discriminant Laplace Approximation
Minyoung Kim, Timothy Hospedales
The meta learning few-shot classification is an emerging problem in machine learning that received enormous attention recently, where the goal is to learn a model that can quickly…
Defensive Tensorization
Adrian Bulat, Jean Kossaifi, Sourav Bhattacharya +5
We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network. The layers of a network are first expressed as…
Self-Supervised Representation Learning: Introduction, Advances and Challenges
Linus Ericsson, Henry Gouk, Chen Change Loy +1
Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation b…
Online Hyperparameter Meta-Learning with Hypergradient Distillation
Hae Beom Lee, Hayeon Lee, Jaewoong Shin +3
Many gradient-based meta-learning methods assume a set of parameters that do not participate in inner-optimization, which can be considered as hyperparameters. Although such hyperp…