An Overview of Multi-Task Learning in Deep Neural Networks

Abstract

Multi-task learning (MTL) has led to successes in many applications of machine learning, from natural language processing and speech recognition to computer vision and drug discovery. This article aims to give a general overview of MTL, particularly in deep neural networks. It introduces the two most common methods for MTL in Deep Learning, gives an overview of the literature, and discusses recent advances. In particular, it seeks to help ML practitioners apply MTL by shedding light on how MTL works and providing guidelines for choosing appropriate auxiliary tasks.

14 pages, 8 figures

References in corpus (3)

• Massively Multitask Networks for Drug Discovery
• Deep Voice: Real-time Neural Text-to-Speech
• Bayesian Multitask Learning with Latent Hierarchies

Cited by in corpus (231)

• Multi-Task Learning with Deep Neural Networks: A Survey
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Regularization for Deep Learning: A Taxonomy
• Earthquake magnitude and location estimation from real time seismic waveforms with a transformer network
• Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation
• WT5?! Training Text-to-Text Models to Explain their Predictions
• CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences
• Multi-task learning for natural language processing in the 2020s: where are we going?
• Multi-modal Deep Analysis for Multimedia
• Automatic Detection of Malware-Generated Domains with Recurrent Neural Models
• Context-Aware Embeddings for Automatic Art Analysis
• Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data
• Reflective Fiber Faults Detection and Characterization Using Long-Short-Term Memory
• Understanding and Improving Information Transfer in Multi-Task Learning
• MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Tasks
• A neural network walks into a lab: towards using deep nets as models for human behavior
• Light Field Image Quality Assessment With Auxiliary Learning Based on Depthwise and Anglewise Separable Convolutions
• Learning to Branch for Multi-Task Learning
• Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• When Deep Learning Meets Multi-Task Learning in SAR ATR: Simultaneous Target Recognition and Segmentation
• A multi-task learning for cavitation detection and cavitation intensity recognition of valve acoustic signals
• DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain
• PADL: Language-Directed Physics-Based Character Control
• Efficiently Identifying Task Groupings for Multi-Task Learning
• Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog
• Vehicle Attribute Recognition by Appearance: Computer Vision Methods for Vehicle Type, Make and Model Classification
• AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving
• Deep Neural Networks for Marine Debris Detection in Sonar Images
• Controllable Multi-Objective Re-ranking with Policy Hypernetworks
• Attention over Parameters for Dialogue Systems
• Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles
• Dynamic Multi-Task Learning for Face Recognition with Facial Expression
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• M2TRec: Metadata-aware Multi-task Transformer for Large-scale and Cold-start free Session-based Recommendations
• AutoML4ETC: Automated Neural Architecture Search for Real-World Encrypted Traffic Classification
• Multi-Modal Multi-Task (3MT) Road Segmentation
• Extending Unsupervised Neural Image Compression With Supervised Multitask Learning
• Multitask Learning for Emotion and Personality Detection
• A Multi-task Learning Framework for Product Ranking with BERT
• Where is the Model Looking At?--Concentrate and Explain the Network Attention
• A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks
• The Next Big Thing(s) in Unsupervised Machine Learning: Five Lessons from Infant Learning
• Implicit Model Specialization through DAG-based Decentralized Federated Learning
• H3DNet: 3D Object Detection Using Hybrid Geometric Primitives
• Advancing Robust Underwater Acoustic Target Recognition through Multi-task Learning and Multi-Gate Mixture-of-Experts
• Unified Mandarin TTS Front-end Based on Distilled BERT Model
• Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings
• A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization
• A New Semi-supervised Learning Benchmark for Classifying View and Diagnosing Aortic Stenosis from Echocardiograms
• Self-Supervised Learning for Medical Image Data with Anatomy-Oriented Imaging Planes
• Learning Cross-Domain Representation with Multi-Graph Neural Network
• Heterogeneous Molecular Graph Neural Networks for Predicting Molecule Properties
• Single-Network Whole-Body Pose Estimation
• Personalized Driver Stress Detection with Multi-task Neural Networks using Physiological Signals
• The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck
• An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
• Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback
• VersatileGait: A Large-Scale Synthetic Gait Dataset with Fine-GrainedAttributes and Complicated Scenarios
• A Deep Probabilistic Model for Customer Lifetime Value Prediction
• Soft Prompt Decoding for Multilingual Dense Retrieval
• Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data
• Generalizing Speaker Verification for Spoof Awareness in the Embedding Space
• A Comprehensive Evaluation of Multi-task Learning and Multi-task Pre-training on EHR Time-series Data
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
• Meta Reinforcement Learning with Task Embedding and Shared Policy
• End-to-end Person Search Sequentially Trained on Aggregated Dataset
• An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks
• Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech
• A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy
• Multi-task learning from fixed-wing UAV images for 2D/3D city modeling
• Context Aware Machine Learning
• Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising
• Deep learning complete intersection Calabi-Yau manifolds
• A Multi-task Learning Framework for Drone State Identification and Trajectory Prediction
• Aggregating From Multiple Target-Shifted Sources
• Uncertainty-Aware Lookahead Factor Models for Quantitative Investing
• Small Towers Make Big Differences
• DaCy: A Unified Framework for Danish NLP
• Predicting the Popularity of Online Videos via Deep Neural Networks
• SpeechNet: A Universal Modularized Model for Speech Processing Tasks
• Closing the gap towards end-to-end autonomous vehicle system
• A Brief Review of Deep Multi-task Learning and Auxiliary Task Learning
• Multitask Learning On Graph Neural Networks Applied To Molecular Property Predictions
• Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
• GDOD: Effective Gradient Descent using Orthogonal Decomposition for Multi-Task Learning
• A Feature Transfer Enabled Multi-Task Deep Learning Model on Medical Imaging
• Outage-Watch: Early Prediction of Outages using Extreme Event Regularizer
• Joint Liver Lesion Segmentation and Classification via Transfer Learning
• Weakly Supervised Multi-task Learning for Concept-based Explainability
• Hierarchical Capsule Prediction Network for Marketing Campaigns Effect
• Transfer Learning in the Field of Renewable Energies -- A Transfer Learning Framework Providing Power Forecasts Throughout the Lifecycle of Wind Farms After Initial Connection to the Electrical Grid
• AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
• Multi-Task Self-Training for Learning General Representations
• HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
• Knowledge Distillation for Multi-task Learning
• Two-path Deep Semi-supervised Learning for Timely Fake News Detection
• An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognition
• Audio-Linguistic Embeddings for Spoken Sentences
• Empirical Study of Multi-Task Hourglass Model for Semantic Segmentation Task
• Auxiliary Task Update Decomposition: The Good, The Bad and The Neutral
• Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension
• Hybrid Model with Time Modeling for Sequential Recommender Systems
• Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
• Comprehensive and Efficient Data Labeling via Adaptive Model Scheduling
• Deep Bayesian Multi-Target Learning for Recommender Systems
• Towards Better Forecasting by Fusing Near and Distant Future Visions
• NeurReg: Neural Registration and Its Application to Image Segmentation
• Bridging the gap between target-based and cell-based drug discovery with a graph generative multi-task model
• Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields
• Reconfigurable Intelligent Surface-Assisted Aerial-Terrestrial Communications via Multi-Task Learning
• Meta-Learning with MAML on Trees
• Multitask Learning for Class-Imbalanced Discourse Classification
• Skin disease diagnosis with deep learning: a review
• Personalized Federated Learning of Driver Prediction Models for Autonomous Driving
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Harvesting the Lyα forest with convolutional neural networks
• Self-supervised learning for fast and scalable time series hyper-parameter tuning
• Frosting Weights for Better Continual Training
• Instance-Level Task Parameters: A Robust Multi-task Weighting Framework
• Invenio: Discovering Hidden Relationships Between Tasks/Domains Using Structured Meta Learning
• Multi-speaker Text-to-speech Synthesis Using Deep Gaussian Processes
• Lossless Compression of Mosaic Images with Convolutional Neural Network Prediction
• Multi-Task Learning with Contextualized Word Representations for Extented Named Entity Recognition
• Cell Type Identification from Single-Cell Transcriptomic Data via Semi-supervised Learning
• Label-efficient audio classification through multitask learning and self-supervision
• Multi-task Learning for Multilingual Neural Machine Translation
• Self-supervised Depth Estimation to Regularise Semantic Segmentation in Knee Arthroscopy
• A Foliated View of Transfer Learning
• Multi-task problems are not multi-objective
• Hypothesis Disparity Regularized Mutual Information Maximization
• Invariant Feature Learning for Sensor-based Human Activity Recognition
• I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch
• Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design
• Balance Regularized Neural Network Models for Causal Effect Estimation
• A Multi-Level Attention Model for Evidence-Based Fact Checking
• GraftNet: An Engineering Implementation of CNN for Fine-grained Multi-label Task
• Supervised Contextual Embeddings for Transfer Learning in Natural Language Processing Tasks
• Deep Homography Estimation for Dynamic Scenes
• TPG-DNN: A Method for User Intent Prediction Based on Total Probability Formula and GRU Loss with Multi-task Learning
• A Simple Approach to Continual Learning by Transferring Skill Parameters
• Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach
• M&M: Multimodal-Multitask Model Integrating Audiovisual Cues in Cognitive Load Assessment
• Taxonomy Completion via Triplet Matching Network
• Hyperparameter Transfer Learning with Adaptive Complexity
• Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network
• A Modularized Neural Network with Language-Specific Output Layers for Cross-lingual Voice Conversion
• Single Shot Multitask Pedestrian Detection and Behavior Prediction
• Multilingual Dialogue Generation with Shared-Private Memory
• Multi-Task Learning via Co-Attentive Sharing for Pedestrian Attribute Recognition
• De-identification of Privacy-related Entities in Job Postings
• 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets
• Multi-Task Learning for Argumentation Mining
• Artificial Intelligence for Pediatric Ophthalmology
• High-Level Perceptual Similarity is Enabled by Learning Diverse Tasks
• Deep Octonion Networks
• From Risk Prediction Models to Risk Assessment Service: A Formulation of Development Paradigm
• FaceLiveNet+: A Holistic Networks For Face Authentication Based On Dynamic Multi-task Convolutional Neural Networks
• Learning Preference-Based Similarities from Face Images using Siamese Multi-Task CNNs
• Graph-Driven Generative Models for Heterogeneous Multi-Task Learning
• A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling
• Doppler Spectrum Classification with CNNs via Heatmap Location Encoding and a Multi-head Output Layer
• Learning an Interpretable Graph Structure in Multi-Task Learning
• Stock Index Prediction with Multi-task Learning and Word Polarity Over Time
• Deep Multi-Task Learning for Cooperative NOMA: System Design and Principles
• Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation
• Towards Robust Pattern Recognition: A Review
• Directional Multivariate Ranking
• Reducing Overlearning through Disentangled Representations by Suppressing Unknown Tasks
• A Biologically Inspired Feature Enhancement Framework for Zero-Shot Learning
• Formant Tracking Using Dilated Convolutional Networks Through Dense Connection with Gating Mechanism
• General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
• Semi-supervised Learning for COVID-19 Image Classification via ResNet
• Network Clustering for Multi-task Learning
• PCA-based Multi Task Learning: a Random Matrix Approach
• Augmenting the User-Item Graph with Textual Similarity Models
• MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces
• Multi-objective optimization and explanation for stroke risk assessment in Shanxi province
• Exceeding the Limits of Visual-Linguistic Multi-Task Learning
• Generative Modeling for Multi-task Visual Learning
• OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction
• Growing Action Spaces
• Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time Series Forecasting
• Learning to Transfer: A Foliated Theory
• How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements
• The Bias-Variance Tradeoff of Doubly Robust Estimator with Targeted $L_1$ regularized Neural Networks Predictions
• Leveraging Commonsense Knowledge on Classifying False News and Determining Checkworthiness of Claims
• VersatileGait: A Large-Scale Synthetic Gait Dataset Towards in-the-Wild Simulation
• Not All Linearizations Are Equally Data-Hungry in Sequence Labeling Parsing
• Predictive Representation Learning for Language Modeling
• Binaural Audio Generation via Multi-task Learning
• Improving Multimodal fusion via Mutual Dependency Maximisation
• YES SIR!Optimizing Semantic Space of Negatives with Self-Involvement Ranker
• Fast Line Search for Multi-Task Learning
• Balancing Average and Worst-case Accuracy in Multitask Learning
• Model Composition: Can Multiple Neural Networks Be Combined into a Single Network Using Only Unlabeled Data?
• Multi-task Recurrent Neural Networks to Simultaneously Infer Mode and Purpose in GPS Trajectories
• Learned Image Compression for Machine Perception
• Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks
• Multi-modal, multi-task, multi-attention (M3) deep learning detection of reticular pseudodrusen: towards automated and accessible classification of age-related macular degeneration
• To Schedule or not to Schedule: Extracting Task Specific Temporal Entities and Associated Negation Constraints
• Sequence-level self-learning with multiple hypotheses
• Logically Consistent Loss for Visual Question Answering
• Distribution Conditional Denoising: A Flexible Discriminative Image Denoiser
• Self-Supervision based Task-Specific Image Collection Summarization
• EXMA: A Genomics Accelerator for Exact-Matching
• Attention-based multi-task learning for speech-enhancement and speaker-identification in multi-speaker dialogue scenario
• CALM: Continuous Adaptive Learning for Language Modeling
• W-net: Simultaneous segmentation of multi-anatomical retinal structures using a multi-task deep neural network
• Interaction Networks: Using a Reinforcement Learner to train other Machine Learning algorithms
• Depth Completion with RGB Prior
• Towards Interpretable Multi-Task Learning Using Bilevel Programming
• Multiple Classification with Split Learning
• Label-Efficient Multi-Task Segmentation using Contrastive Learning
• Fair Meta-Learning For Few-Shot Classification
• Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification With K-means Features
• Dense Fusion Classmate Network for Land Cover Classification
• On the relationship between multitask neural networks and multitask Gaussian Processes
• Improve SGD Training via Aligning Mini-batches
• MEETING BOT: Reinforcement Learning for Dialogue Based Meeting Scheduling
• Towards end-to-end pulsed eddy current classification and regression with CNN
• Cooperative Embeddings for Instance, Attribute and Category Retrieval
• Multi-task Learning for Chinese Word Usage Errors Detection
• Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
• Low-Rank Deep Convolutional Neural Network for Multi-Task Learning
• Predicting Auditory Spatial Attention from EEG using Single- and Multi-task Convolutional Neural Networks
• Coded-InvNet for Resilient Prediction Serving Systems