GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
arXiv:2102.07868
Abstract
Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the Pólya Gamma augmentation scheme. As a result, our method scales well with both the number of classes and data size. We demonstrate the effectiveness of our method against other Gaussian process training baselines, and we show how our general GP approach achieves improved accuracy on standard incremental few-shot learning benchmarks.
References in corpus (15)
- Overcoming catastrophic forgetting in neural networks
- Prototypical Networks for Few-shot Learning
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
- Scalable Variational Gaussian Process Classification
- Progress & Compress: A scalable framework for continual learning
- Overcoming Catastrophic Forgetting by Incremental Moment Matching
- Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
- Stochastic Variational Deep Kernel Learning
- Functional Regularisation for Continual Learning with Gaussian Processes
- Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation
- Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling
- GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
- Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition
- Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes