Measuring and Harnessing Transference in Multi-Task Learning
arXiv:2010.15413
Abstract
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in particular, identifying the tasks that would benefit from co-training remains a challenging design question. In this paper, we analyze the dynamics of information transfer, or transference, across tasks throughout training. Specifically, we develop a similarity measure that can quantify transference among tasks and use this quantity to both better understand the optimization dynamics of multi-task learning as well as improve overall learning performance. In the latter case, we propose two methods to leverage our transference metric. The first operates at a macro-level by selecting which tasks should train together while the second functions at a micro-level by determining how to combine task gradients at each training step. We find these methods can lead to significant improvement over prior work on three supervised multi-task learning benchmarks and one multi-task reinforcement learning paradigm.
References in corpus (25)
- Adam: A Method for Stochastic Optimization
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- An Overview of Multi-Task Learning in Deep Neural Networks
- A Survey on Multi-Task Learning
- On First-Order Meta-Learning Algorithms
- GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
- Lookahead Optimizer: k steps forward, 1 step back
- Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
- Multi-Task Learning as Multi-Objective Optimization
- Averaging Weights Leads to Wider Optima and Better Generalization
- Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
- Bayesian Model-Agnostic Meta-Learning
- AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning
- Gradient Surgery for Multi-Task Learning
- Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models
- Trace Norm Regularised Deep Multi-Task Learning
- Pareto Multi-Task Learning
- Learning to Branch for Multi-Task Learning
- Regularizing Deep Multi-Task Networks using Orthogonal Gradients
- Identifying beneficial task relations for multi-task learning in deep neural networks
- Small Towers Make Big Differences
- On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment
- Chameleon: Learning Model Initializations Across Tasks With Different Schemas