An Overview of Multi-Task Learning in Deep Neural Networks

Abstract

Multi-task learning (MTL) has led to successes in many applications of machine learning, from natural language processing and speech recognition to computer vision and drug discovery. This article aims to give a general overview of MTL, particularly in deep neural networks. It introduces the two most common methods for MTL in Deep Learning, gives an overview of the literature, and discusses recent advances. In particular, it seeks to help ML practitioners apply MTL by shedding light on how MTL works and providing guidelines for choosing appropriate auxiliary tasks.

14 pages, 8 figures

References in corpus (3)

• Massively Multitask Networks for Drug Discovery
• Deep Voice: Real-time Neural Text-to-Speech
• Bayesian Multitask Learning with Latent Hierarchies

Cited by in corpus (213)

• Multi-Task Learning with Deep Neural Networks: A Survey
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Regularization for Deep Learning: A Taxonomy
• Earthquake magnitude and location estimation from real time seismic waveforms with a transformer network
• Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation
• WT5?! Training Text-to-Text Models to Explain their Predictions
• CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences
• Multi-task learning for natural language processing in the 2020s: where are we going?
• Automatic Detection of Malware-Generated Domains with Recurrent Neural Models
• Multi-modal Deep Analysis for Multimedia
• Context-Aware Embeddings for Automatic Art Analysis
• Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data
• Reflective Fiber Faults Detection and Characterization Using Long-Short-Term Memory
• A neural network walks into a lab: towards using deep nets as models for human behavior
• MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Tasks
• Understanding and Improving Information Transfer in Multi-Task Learning
• Learning to Branch for Multi-Task Learning
• Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection
• A multi-task learning for cavitation detection and cavitation intensity recognition of valve acoustic signals
• DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain
• Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog
• Efficiently Identifying Task Groupings for Multi-Task Learning
• AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving
• Vehicle Attribute Recognition by Appearance: Computer Vision Methods for Vehicle Type, Make and Model Classification
• Deep Neural Networks for Marine Debris Detection in Sonar Images
• Attention over Parameters for Dialogue Systems
• Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles
• Dynamic Multi-Task Learning for Face Recognition with Facial Expression
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• M2TRec: Metadata-aware Multi-task Transformer for Large-scale and Cold-start free Session-based Recommendations
• A Multi-task Learning Framework for Product Ranking with BERT
• Multitask Learning for Emotion and Personality Detection
• Extending Unsupervised Neural Image Compression With Supervised Multitask Learning
• A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks
• Where is the Model Looking At?--Concentrate and Explain the Network Attention
• The Next Big Thing(s) in Unsupervised Machine Learning: Five Lessons from Infant Learning
• Implicit Model Specialization through DAG-based Decentralized Federated Learning
• H3DNet: 3D Object Detection Using Hybrid Geometric Primitives
• Unified Mandarin TTS Front-end Based on Distilled BERT Model
• A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization
• Learning Cross-Domain Representation with Multi-Graph Neural Network
• Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings
• A New Semi-supervised Learning Benchmark for Classifying View and Diagnosing Aortic Stenosis from Echocardiograms
• Heterogeneous Molecular Graph Neural Networks for Predicting Molecule Properties
• Personalized Driver Stress Detection with Multi-task Neural Networks using Physiological Signals
• Single-Network Whole-Body Pose Estimation
• The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck
• Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback
• An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
• A Deep Probabilistic Model for Customer Lifetime Value Prediction
• VersatileGait: A Large-Scale Synthetic Gait Dataset with Fine-GrainedAttributes and Complicated Scenarios
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
• A Comprehensive Evaluation of Multi-task Learning and Multi-task Pre-training on EHR Time-series Data
• End-to-end Person Search Sequentially Trained on Aggregated Dataset
• An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
• Meta Reinforcement Learning with Task Embedding and Shared Policy
• Multi-task learning from fixed-wing UAV images for 2D/3D city modeling
• A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy
• Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech
• Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising
• Uncertainty-Aware Lookahead Factor Models for Quantitative Investing
• Aggregating From Multiple Target-Shifted Sources
• Context Aware Machine Learning
• DaCy: A Unified Framework for Danish NLP
• SpeechNet: A Universal Modularized Model for Speech Processing Tasks
• Predicting the Popularity of Online Videos via Deep Neural Networks
• Closing the gap towards end-to-end autonomous vehicle system
• Small Towers Make Big Differences
• A Brief Review of Deep Multi-task Learning and Auxiliary Task Learning
• Two-path Deep Semi-supervised Learning for Timely Fake News Detection
• HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
• Hierarchical Capsule Prediction Network for Marketing Campaigns Effect
• Multi-Task Self-Training for Learning General Representations
• Knowledge Distillation for Multi-task Learning
• Transfer Learning in the Field of Renewable Energies -- A Transfer Learning Framework Providing Power Forecasts Throughout the Lifecycle of Wind Farms After Initial Connection to the Electrical Grid
• Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
• AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
• Multitask Learning On Graph Neural Networks Applied To Molecular Property Predictions
• Auxiliary Task Update Decomposition: The Good, The Bad and The Neutral
• Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension
• Joint Liver Lesion Segmentation and Classification via Transfer Learning
• An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognition
• Empirical Study of Multi-Task Hourglass Model for Semantic Segmentation Task
• A Feature Transfer Enabled Multi-Task Deep Learning Model on Medical Imaging
• Audio-Linguistic Embeddings for Spoken Sentences
• Weakly Supervised Multi-task Learning for Concept-based Explainability
• Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields
• Skin disease diagnosis with deep learning: a review
• Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
• Personalized Federated Learning of Driver Prediction Models for Autonomous Driving
• NeurReg: Neural Registration and Its Application to Image Segmentation
• Multitask Learning for Class-Imbalanced Discourse Classification
• Bridging the gap between target-based and cell-based drug discovery with a graph generative multi-task model
• Deep Bayesian Multi-Target Learning for Recommender Systems
• Reconfigurable Intelligent Surface-Assisted Aerial-Terrestrial Communications via Multi-Task Learning
• Comprehensive and Efficient Data Labeling via Adaptive Model Scheduling
• Meta-Learning with MAML on Trees
• Hybrid Model with Time Modeling for Sequential Recommender Systems
• Towards Better Forecasting by Fusing Near and Distant Future Visions
• Instance-Level Task Parameters: A Robust Multi-task Weighting Framework
• Multi-Task Learning with Contextualized Word Representations for Extented Named Entity Recognition
• Lossless Compression of Mosaic Images with Convolutional Neural Network Prediction
• Frosting Weights for Better Continual Training
• Invenio: Discovering Hidden Relationships Between Tasks/Domains Using Structured Meta Learning
• Label-efficient audio classification through multitask learning and self-supervision
• Multi-task Learning for Multilingual Neural Machine Translation
• Multi-speaker Text-to-speech Synthesis Using Deep Gaussian Processes
• A Foliated View of Transfer Learning
• Self-supervised Depth Estimation to Regularise Semantic Segmentation in Knee Arthroscopy
• Cell Type Identification from Single-Cell Transcriptomic Data via Semi-supervised Learning
• Self-supervised learning for fast and scalable time series hyper-parameter tuning
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Harvesting the Lyα forest with convolutional neural networks
• Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network
• Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach
• Deep Homography Estimation for Dynamic Scenes
• Multi-task problems are not multi-objective
• Hypothesis Disparity Regularized Mutual Information Maximization
• Invariant Feature Learning for Sensor-based Human Activity Recognition
• I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch
• A Simple Approach to Continual Learning by Transferring Skill Parameters
• Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design
• Balance Regularized Neural Network Models for Causal Effect Estimation
• A Multi-Level Attention Model for Evidence-Based Fact Checking
• Taxonomy Completion via Triplet Matching Network
• Hyperparameter Transfer Learning with Adaptive Complexity
• Multilingual Dialogue Generation with Shared-Private Memory
• Supervised Contextual Embeddings for Transfer Learning in Natural Language Processing Tasks
• GraftNet: An Engineering Implementation of CNN for Fine-grained Multi-label Task
• De-identification of Privacy-related Entities in Job Postings
• A Modularized Neural Network with Language-Specific Output Layers for Cross-lingual Voice Conversion
• Multi-Task Learning via Co-Attentive Sharing for Pedestrian Attribute Recognition
• Single Shot Multitask Pedestrian Detection and Behavior Prediction
• TPG-DNN: A Method for User Intent Prediction Based on Total Probability Formula and GRU Loss with Multi-task Learning
• A Biologically Inspired Feature Enhancement Framework for Zero-Shot Learning
• Reducing Overlearning through Disentangled Representations by Suppressing Unknown Tasks
• Directional Multivariate Ranking
• Semi-supervised Learning for COVID-19 Image Classification via ResNet
• Towards Robust Pattern Recognition: A Review
• Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation
• Network Clustering for Multi-task Learning
• Deep Multi-Task Learning for Cooperative NOMA: System Design and Principles
• Stock Index Prediction with Multi-task Learning and Word Polarity Over Time
• Learning an Interpretable Graph Structure in Multi-Task Learning
• PCA-based Multi Task Learning: a Random Matrix Approach
• Augmenting the User-Item Graph with Textual Similarity Models
• MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces
• Doppler Spectrum Classification with CNNs via Heatmap Location Encoding and a Multi-head Output Layer
• Multi-objective optimization and explanation for stroke risk assessment in Shanxi province
• A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling
• Exceeding the Limits of Visual-Linguistic Multi-Task Learning
• Generative Modeling for Multi-task Visual Learning
• Graph-Driven Generative Models for Heterogeneous Multi-Task Learning
• OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction
• Learning Preference-Based Similarities from Face Images using Siamese Multi-Task CNNs
• FaceLiveNet+: A Holistic Networks For Face Authentication Based On Dynamic Multi-task Convolutional Neural Networks
• From Risk Prediction Models to Risk Assessment Service: A Formulation of Development Paradigm
• Deep Octonion Networks
• High-Level Perceptual Similarity is Enabled by Learning Diverse Tasks
• 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets
• Artificial Intelligence for Pediatric Ophthalmology
• Multi-Task Learning for Argumentation Mining
• Formant Tracking Using Dilated Convolutional Networks Through Dense Connection with Gating Mechanism
• General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
• Binaural Audio Generation via Multi-task Learning
• Depth Completion with RGB Prior
• EXMA: A Genomics Accelerator for Exact-Matching
• Attention-based multi-task learning for speech-enhancement and speaker-identification in multi-speaker dialogue scenario
• Not All Linearizations Are Equally Data-Hungry in Sequence Labeling Parsing
• Predicting Auditory Spatial Attention from EEG using Single- and Multi-task Convolutional Neural Networks
• Leveraging Commonsense Knowledge on Classifying False News and Determining Checkworthiness of Claims
• The Bias-Variance Tradeoff of Doubly Robust Estimator with Targeted $L_1$ regularized Neural Networks Predictions
• W-net: Simultaneous segmentation of multi-anatomical retinal structures using a multi-task deep neural network
• Sequence-level self-learning with multiple hypotheses
• Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
• Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification With K-means Features
• Learning to Transfer: A Foliated Theory
• Dense Fusion Classmate Network for Land Cover Classification
• Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time Series Forecasting
• Low-Rank Deep Convolutional Neural Network for Multi-Task Learning
• Coded-InvNet for Resilient Prediction Serving Systems
• Cooperative Embeddings for Instance, Attribute and Category Retrieval
• CALM: Continuous Adaptive Learning for Language Modeling
• On the relationship between multitask neural networks and multitask Gaussian Processes
• Growing Action Spaces
• How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements
• Interaction Networks: Using a Reinforcement Learner to train other Machine Learning algorithms
• Multi-task Learning for Chinese Word Usage Errors Detection
• VersatileGait: A Large-Scale Synthetic Gait Dataset Towards in-the-Wild Simulation
• Improve SGD Training via Aligning Mini-batches
• Distribution Conditional Denoising: A Flexible Discriminative Image Denoiser
• Logically Consistent Loss for Visual Question Answering
• To Schedule or not to Schedule: Extracting Task Specific Temporal Entities and Associated Negation Constraints
• Multi-modal, multi-task, multi-attention (M3) deep learning detection of reticular pseudodrusen: towards automated and accessible classification of age-related macular degeneration
• Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks
• Multiple Classification with Split Learning
• Learned Image Compression for Machine Perception
• Predictive Representation Learning for Language Modeling
• Label-Efficient Multi-Task Segmentation using Contrastive Learning
• Multi-task Recurrent Neural Networks to Simultaneously Infer Mode and Purpose in GPS Trajectories
• Model Composition: Can Multiple Neural Networks Be Combined into a Single Network Using Only Unlabeled Data?
• Towards Interpretable Multi-Task Learning Using Bilevel Programming
• Self-Supervision based Task-Specific Image Collection Summarization
• Balancing Average and Worst-case Accuracy in Multitask Learning
• Fast Line Search for Multi-Task Learning
• MEETING BOT: Reinforcement Learning for Dialogue Based Meeting Scheduling
• YES SIR!Optimizing Semantic Space of Negatives with Self-Involvement Ranker
• Improving Multimodal fusion via Mutual Dependency Maximisation
• Towards end-to-end pulsed eddy current classification and regression with CNN
• Fair Meta-Learning For Few-Shot Classification