Probabilistic task modelling for meta-learning

Abstract

We propose probabilistic task modelling -- a generative probabilistic model for collections of tasks used in meta-learning. The proposed model combines variational auto-encoding and latent Dirichlet allocation to model each task as a mixture of Gaussian distribution in an embedding space. Such modelling provides an explicit representation of a task through its task-theme mixture. We present an efficient approximation inference technique based on variational inference method for empirical Bayes parameter estimation. We perform empirical evaluations to validate the task uncertainty and task distance produced by the proposed method through correlation diagrams of the prediction accuracy on testing tasks. We also carry out experiments of task selection in meta-learning to demonstrate how the task relatedness inferred from the proposed model help to facilitate meta-learning algorithms.

Correct typos, notations and update related work

References in corpus (6)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Clustered Multi-Task Learning: A Convex Formulation
• Infinite Mixture Prototypes for Few-Shot Learning
• LEEP: A New Measure to Evaluate Transferability of Learned Representations
• The continuous Bernoulli: fixing a pervasive error in variational autoencoders