Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes

Abstract

Few-shot classification (FSC), the task of adapting a classifier to unseen classes given a small labeled dataset, is an important step on the path toward human-like machine learning. Bayesian methods are well-suited to tackling the fundamental issue of overfitting in the few-shot scenario because they allow practitioners to specify prior beliefs and update those beliefs in light of observed data. Contemporary approaches to Bayesian few-shot classification maintain a posterior distribution over model parameters, which is slow and requires storage that scales with model size. Instead, we propose a Gaussian process classifier based on a novel combination of Pólya-Gamma augmentation and the one-vs-each softmax approximation that allows us to efficiently marginalize over functions rather than model parameters. We demonstrate improved accuracy and uncertainty quantification on both standard few-shot classification benchmarks and few-shot domain transfer tasks.

Extended version of accepted ICLR 2021 submission. 34 pages, 9 figures

References in corpus (8)

• On Calibration of Modern Neural Networks
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Functional Variational Bayesian Neural Networks
• Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation
• One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities
• Sampling Polya-Gamma random variates: alternate and approximate techniques
• Stochastic Prototype Embeddings
• Information Theoretic Meta Learning with Gaussian Processes

Cited by in corpus (3)

• GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
• Non-Gaussian Gaussian Processes for Few-Shot Regression
• How Sensitive are Meta-Learners to Dataset Imbalance?