Multi-Task Learning with Deep Neural Networks: A Survey
arXiv:2009.09796
Abstract
Multi-task learning (MTL) is a subfield of machine learning in which multiple tasks are simultaneously learned by a shared model. Such approaches offer advantages like improved data efficiency, reduced overfitting through shared representations, and fast learning by leveraging auxiliary information. However, the simultaneous learning of multiple tasks presents new design and optimization challenges, and choosing which tasks should be learned jointly is in itself a non-trivial problem. In this survey, we give an overview of multi-task learning methods for deep neural networks, with the aim of summarizing both the well-established and most recent directions within the field. Our discussion is structured according to a partition of the existing deep MTL techniques into three groups: architectures, optimization methods, and task relationship learning. We also provide a summary of common multi-task benchmarks.
References in corpus (22)
- Sequence to Sequence Learning with Neural Networks
- Natural Language Processing (almost) from Scratch
- Neural Architecture Search with Reinforcement Learning
- An Overview of Multi-Task Learning in Deep Neural Networks
- Microsoft COCO Captions: Data Collection and Evaluation Server
- PathNet: Evolution Channels Gradient Descent in Super Neural Networks
- Solving Rubik's Cube with a Robot Hand
- One Model To Learn Them All
- Multi-Task Deep Neural Networks for Natural Language Understanding
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
- Learning and Transfer of Modulated Locomotor Controllers
- Pareto Multi-Task Learning
- Routing Networks and the Challenges of Modular and Compositional Computation
- Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
- Deep Multi-Task Learning with Shared Memory
- Deep Model Transferability from Attribution Maps
- Learning Sparse Sharing Architectures for Multiple Tasks
- Identifying beneficial task relations for multi-task learning in deep neural networks
- Meta Reinforcement Learning with Task Embedding and Shared Policy
- Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis
- Learning Functions to Study the Benefit of Multitask Learning
- Transfer Learning to Learn with Multitask Neural Model Search