PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior
arXiv:2102.03748
Abstract
By leveraging experience from previous tasks, meta-learning algorithms can achieve effective fast adaptation ability when encountering new tasks. However it is unclear how the generalization property applies to new tasks. Probably approximately correct (PAC) Bayes bound theory provides a theoretical framework to analyze the generalization performance for meta-learning. We derive three novel generalisation error bounds for meta-learning based on PAC-Bayes relative entropy bound. Furthermore, using the empirical risk minimization (ERM) method, a PAC-Bayes bound for meta-learning with data-dependent prior is developed. Experiments illustrate that the proposed three PAC-Bayes bounds for meta-learning guarantee a competitive generalization performance guarantee, and the extended PAC-Bayes bound with data-dependent prior can achieve rapid convergence ability.
References in corpus (15)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- A Simple Neural Attentive Meta-Learner
- Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
- Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
- A PAC-Bayesian bound for Lifelong Learning
- Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
- Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach
- Alpha MAML: Adaptive Model-Agnostic Meta-Learning
- A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent
- Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience
- Tighter risk certificates for neural networks
- On the role of data in PAC-Bayes bounds
- Meta-Learning PAC-Bayes Priors in Model Averaging
- PAC-Bayesian Transportation Bound