Disentangling Transfer and Interference in Multi-Domain Learning

Abstract

Humans are incredibly good at transferring knowledge from one domain to another, enabling rapid learning of new tasks. Likewise, transfer learning has enabled enormous success in many computer vision problems using pretraining. However, the benefits of transfer in multi-domain learning, where a network learns multiple tasks defined by different datasets, has not been adequately studied. Learning multiple domains could be beneficial, or these domains could interfere with each other given limited network capacity. Understanding how deep neural networks of varied capacity facilitate transfer across inputs from different distributions is a critical step towards open world learning. In this work, we decipher the conditions where interference and knowledge transfer occur in multi-domain learning. We propose new metrics disentangling interference and transfer, set up experimental protocols, and examine the roles of network capacity, task grouping, and dynamic loss weighting in reducing interference and facilitating transfer.

arXiv commentAAAI 2022 PracticalDL Workshop

References in corpus (18)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Visual Domain Adaptation: A Survey
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Learning multiple visual domains with residual adapters
• Three scenarios for continual learning
• Similarity of Neural Network Representations Revisited
• Multi-Task Learning as Multi-Objective Optimization
• A Comprehensive Survey on Transfer Learning
• Which Tasks Should Be Learned Together in Multi-task Learning?
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• Learning Multiple Tasks with Multilinear Relationship Networks
• Auxiliary Tasks in Multi-task Learning
• Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
• Universal representations:The missing link between faces, text, planktons, and cat breeds
• Understanding and Improving Information Transfer in Multi-Task Learning
• Budget-Aware Adapters for Multi-Domain Learning
• Scaling Laws for Transfer
• Self-Paced Multi-Task Learning