Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning

Abstract

We propose a novel adaptive transfer learning framework, learning to transfer learn (L2TL), to improve performance on a target dataset by careful extraction of the related information from a source dataset. Our framework considers cooperative optimization of shared weights between models for source and target tasks, and adjusts the constituent loss weights adaptively. The adaptation of the weights is based on a reinforcement learning (RL) selection policy, guided with a performance metric on the target validation set. We demonstrate that L2TL outperforms fine-tuning baselines and other adaptive transfer learning methods on eight datasets. In the regimes of small-scale target datasets and significant label mismatch between source and target datasets, L2TL shows particularly large benefits.

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Neural Architecture Search with Reinforcement Learning
• WaveNet: A Generative Model for Raw Audio
• Learning to Reweight Examples for Robust Deep Learning
• AutoAugment: Learning Augmentation Policies from Data
• Deep Learning Scaling is Predictable, Empirically
• Data Shapley: Equitable Valuation of Data for Machine Learning
• Domain Adaptive Transfer Learning with Specialist Models
• Towards Understanding the Transferability of Deep Representations