Trace Norm Regularised Deep Multi-Task Learning

Abstract

We propose a framework for training multiple neural networks simultaneously. The parameters from all models are regularised by the tensor trace norm, so that each neural network is encouraged to reuse others' parameters if possible -- this is the main motivation behind multi-task learning. In contrast to many deep multi-task learning models, we do not predefine a parameter sharing strategy by specifying which layers have tied parameters. Instead, our framework considers sharing for all shareable layers, and the sharing strategy is learned in a data-driven way.

Submission to Workshop track - ICLR 2017

References in corpus (1)

• Deep Multi-task Representation Learning: A Tensor Factorisation Approach

Cited by in corpus (27)

• Multi-Task Learning with Deep Neural Networks: A Survey
• Gradient Surgery for Multi-Task Learning
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• Pedestrian Attribute Recognition: A Survey
• Efficiently Identifying Task Groupings for Multi-Task Learning
• Multi-Task Graph Autoencoders
• Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference
• Single-Network Whole-Body Pose Estimation
• Multi-Task Learning by Deep Collaboration and Application in Facial Landmark Detection
• Measuring and Harnessing Transference in Multi-Task Learning
• Auxiliary Learning for Deep Multi-task Learning
• Multi-Task Image-Based Dietary Assessment for Food Recognition and Portion Size Estimation
• Deep Bayesian Multi-Target Learning for Recommender Systems
• Joint Detection of Malicious Domains and Infected Clients
• Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer
• Tasks Structure Regularization in Multi-Task Learning for Improving Facial Attribute Prediction
• GraftNet: An Engineering Implementation of CNN for Fine-grained Multi-label Task
• Tackling Ordinal Regression Problem for Heterogeneous Data: Sparse and Deep Multi-Task Learning Approaches
• Network Clustering for Multi-task Learning
• A Biologically Inspired Feature Enhancement Framework for Zero-Shot Learning
• Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing
• Boosting Supervised Learning Performance with Co-training
• Exploring Correlations in Multiple Facial Attributes through Graph Attention Network
• Towards All-around Knowledge Transferring: Learning From Task-irrelevant Labels
• Unifying Multi-Domain Multi-Task Learning: Tensor and Neural Network Perspectives
• Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
• Exploring Data Aggregation and Transformations to Generalize across Visual Domains