Identifying beneficial task relations for multi-task learning in deep neural networks

Abstract

Multi-task learning (MTL) in deep neural networks for NLP has recently received increasing interest due to some compelling benefits, including its potential to efficiently regularize models and to reduce the need for labeled data. While it has brought significant improvements in a number of NLP tasks, mixed results have been reported, and little is known about the conditions under which MTL leads to gains in NLP. This paper sheds light on the specific task relations that can lead to gains from MTL models over single-task setups.

Accepted for publication at EACL 2017

References in corpus (2)

• How Transferable are Neural Networks in NLP Applications?
• Keystroke dynamics as signal for shallow syntactic parsing

Cited by in corpus (8)

• Multi-Task Learning with Deep Neural Networks: A Survey
• Measuring and Harnessing Transference in Multi-Task Learning
• AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning
• Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification
• HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxiliary Tasks
• Learning Representations of Social Media Users
• Hierarchical Multitask Learning Approach for BERT
• Concurrent Learning of Semantic Relations