Learning to Learn: Meta-Critic Networks for Sample Efficient Learning

Abstract

We propose a novel and flexible approach to meta-learning for learning-to-learn from only a few examples. Our framework is motivated by actor-critic reinforcement learning, but can be applied to both reinforcement and supervised learning. The key idea is to learn a meta-critic: an action-value function neural network that learns to criticise any actor trying to solve any specified task. For supervised learning, this corresponds to the novel idea of a trainable task-parametrised loss generator. This meta-critic approach provides a route to knowledge transfer that can flexibly deal with few-shot and semi-supervised conditions for both reinforcement and supervised learning. Promising results are shown on both reinforcement and supervised learning problems.

Technical report, 12 pages, 3 figures, 2 tables

References in corpus (5)

• Distilling the Knowledge in a Neural Network
• Matching Networks for One Shot Learning
• Learning feed-forward one-shot learners
• Learning Parameterized Skills
• Generalizing Skills with Semi-Supervised Reinforcement Learning

Cited by in corpus (34)

• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Transferring Robustness for Graph Neural Network Against Poisoning Attacks
• Meta-learning PINN loss functions
• Learning to Generalize: Meta-Learning for Domain Generalization
• Assessing Generalization in Deep Reinforcement Learning
• Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
• Feature-Critic Networks for Heterogeneous Domain Generalization
• VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives
• Unsupervised Meta-Learning for Reinforcement Learning
• Learning from Few Samples: A Survey
• Online Meta-Critic Learning for Off-Policy Actor-Critic Methods
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
• Graph Inference Learning for Semi-supervised Classification
• BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
• From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)
• Instance Credibility Inference for Few-Shot Learning
• Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• Guiding Policies with Language via Meta-Learning
• Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
• Bottom-Up Meta-Policy Search
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning
• Meta-Learning with Network Pruning
• A Heuristic Search Algorithm Using the Stability of Learning Algorithms in Certain Scenarios as the Fitness Function: An Artificial General Intelligence Engineering Approach
• Meta-Reinforcement Learning for Heuristic Planning
• Recomposing the Reinforcement Learning Building Blocks with Hypernetworks
• Effective Regularization Through Loss-Function Metalearning
• Subjectivity Learning Theory towards Artificial General Intelligence
• Metabolize Neural Network
• HMRL: Hyper-Meta Learning for Sparse Reward Reinforcement Learning Problem
• Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition
• Transfer Reinforcement Learning across Homotopy Classes