Meta-learning for mixed linear regression
arXiv:2002.08936
Abstract
In modern supervised learning, there are a large number of tasks, but many of them are associated with only a small amount of labeled data. These include data from medical image processing and robotic interaction. Even though each individual task cannot be meaningfully trained in isolation, one seeks to meta-learn across the tasks from past experiences by exploiting some similarities. We study a fundamental question of interest: When can abundant tasks with small data compensate for lack of tasks with big data? We focus on a canonical scenario where each task is drawn from a mixture of linear regressions, and identify sufficient conditions for such a graceful exchange to hold; The total number of examples necessary with only small data tasks scales similarly as when big data tasks are available. To this end, we introduce a novel spectral approach and show that we can efficiently utilize small data tasks with the help of medium data tasks each with examples.
References in corpus (14)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Theoretical Models of Learning to Learn
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- Meta-Learning with Implicit Gradients
- Meta-Learning with Latent Embedding Optimization
- Fast Context Adaptation via Meta-Learning
- Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
- Bayesian Model-Agnostic Meta-Learning
- Deep Meta-Learning: Learning to Learn in the Concept Space
- Random design analysis of ridge regression
- Spectral Experts for Estimating Mixtures of Linear Regressions
- Solving a Mixture of Many Random Linear Equations by Tensor Decomposition and Alternating Minimization
- Maximum Likelihood Estimation for Learning Populations of Parameters
- Sublinear Optimal Policy Value Estimation in Contextual Bandits