ALPaCA vs. GP-based Prior Learning: A Comparison between two Bayesian Meta-Learning Algorithms

Abstract

Meta-learning or few-shot learning, has been successfully applied in a wide range of domains from computer vision to reinforcement learning. Among the many frameworks proposed for meta-learning, bayesian methods are particularly favoured when accurate and calibrated uncertainty estimate is required. In this paper, we investigate the similarities and disparities among two recently published bayesian meta-learning methods: ALPaCA (Harrison et al. [2018]) and PACOH (Rothfuss et al. [2020]). We provide theoretical analysis as well as empirical benchmarks across synthetic and real-world dataset. While ALPaCA holds advantage in computation time by the usage of a linear kernel, general GP-based methods provide much more flexibility and achieves better result across datasets when using a common kernel such as SE (Squared Exponential) kernel. The influence of different loss function choice is also discussed.

References in corpus (6)

• Decoupled Weight Decay Regularization
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• Safe Model-based Reinforcement Learning with Stability Guarantees
• Meta-Learning: A Survey
• Investigating Human Priors for Playing Video Games
• Reducing Drift in Visual Odometry by Inferring Sun Direction Using a Bayesian Convolutional Neural Network

Cited by in corpus (2)

• Meta-Learning Conjugate Priors for Few-Shot Bayesian Optimization
• Bayesian Meta-Learning Through Variational Gaussian Processes