Non-Gaussian Gaussian Processes for Few-Shot Regression
arXiv:2110.13561
Abstract
Gaussian Processes (GPs) have been widely used in machine learning to model distributions over functions, with applications including multi-modal regression, time-series prediction, and few-shot learning. GPs are particularly useful in the last application since they rely on Normal distributions and enable closed-form computation of the posterior probability function. Unfortunately, because the resulting posterior is not flexible enough to capture complex distributions, GPs assume high similarity between subsequent tasks - a requirement rarely met in real-world conditions. In this work, we address this limitation by leveraging the flexibility of Normalizing Flows to modulate the posterior predictive distribution of the GP. This makes the GP posterior locally non-Gaussian, therefore we name our method Non-Gaussian Gaussian Processes (NGGPs). More precisely, we propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. We empirically tested the flexibility of NGGPs on various few-shot learning regression datasets, showing that the mapping can incorporate context embedding information to model different noise levels for periodic functions. As a result, our method shares the structure of the problem between subsequent tasks, but the contextualization allows for adaptation to dissimilarities. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
References in corpus (19)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Variational Inference with Normalizing Flows
- Matching Networks for One Shot Learning
- Density estimation using Real NVP
- A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
- Gaussian Process Kernels for Pattern Discovery and Extrapolation
- A Closer Look at Few-shot Classification
- Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Gaussian Process Regression for Probabilistic Short-term Solar Output Forecast
- Meta-Learning without Memorization
- Probabilistic Model-Agnostic Meta-Learning
- Meta-Learning Mean Functions for Gaussian Processes
- Adaptive Deep Kernel Learning
- Kernel Dependence Regularizers and Gaussian Processes with Applications to Algorithmic Fairness
- Reconciling meta-learning and continual learning with online mixtures of tasks
- Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes
- Task-similarity Aware Meta-learning through Nonparametric Kernel Regression
- Information Theoretic Meta Learning with Gaussian Processes