A Survey of Deep Meta-Learning

Abstract

Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into i)~metric-, ii)~model-, and iii)~optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.

Published in the AI Review (AIRE) Journal (2021)

References in corpus (1)

• OpenML: networked science in machine learning

Cited by in corpus (17)

• A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge
• Machine learning for industrial sensing and control: A survey and practical perspective
• Embrace Limited and Imperfect Training Datasets: Opportunities and Challenges in Plant Disease Recognition Using Deep Learning
• A Tutorial on Meta-Reinforcement Learning
• Fine-Grained Trajectory-based Travel Time Estimation for Multi-city Scenarios Based on Deep Meta-Learning
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• Sharing to learn and learning to share; Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta review
• Accelerating physics-informed neural network based 1D arc simulation by meta learning
• Meta-learning in healthcare: A survey
• Knowledge-integrated AutoEncoder Model
• A Generalised Deep Meta-Learning Model for Automated Quality Control of Cardiovascular Magnetic Resonance Images
• Stateless Neural Meta-Learning using Second-Order Gradients
• Multi-layer Representation Learning for Robust OOD Image Classification
• Principal component analysis for Gaussian process posteriors
• CNN Feature Map Augmentation for Single-Source Domain Generalization
• Knowledge-enhanced Agents for Interactive Text Games
• Benchmarking for Practice: Few-Shot Time-Series Crop-Type Classification on the EuroCropsML Dataset