Budget-Aware Adapters for Multi-Domain Learning
arXiv:1905.06242 · doi:10.1109/ICCV.2019.00047
Abstract
Multi-Domain Learning (MDL) refers to the problem of learning a set of models derived from a common deep architecture, each one specialized to perform a task in a certain domain (e.g., photos, sketches, paintings). This paper tackles MDL with a particular interest in obtaining domain-specific models with an adjustable budget in terms of the number of network parameters and computational complexity. Our intuition is that, as in real applications the number of domains and tasks can be very large, an effective MDL approach should not only focus on accuracy but also on having as few parameters as possible. To implement this idea we derive specialized deep models for each domain by adapting a pre-trained architecture but, differently from other methods, we propose a novel strategy to automatically adjust the computational complexity of the network. To this aim, we introduce Budget-Aware Adapters that select the most relevant feature channels to better handle data from a novel domain. Some constraints on the number of active switches are imposed in order to obtain a network respecting the desired complexity budget. Experimentally, we show that our approach leads to recognition accuracy competitive with state-of-the-art approaches but with much lighter networks both in terms of storage and computation.
arXiv commentICCV 2019
References in corpus (7)
- Overcoming catastrophic forgetting in neural networks
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Fine-Grained Visual Classification of Aircraft
- Progressive Neural Networks
- Learning multiple visual domains with residual adapters
- Slimmable Neural Networks
- Fast Decoding in Sequence Models using Discrete Latent Variables
Cited by in corpus (7)
- Boosting Binary Masks for Multi-Domain Learning through Affine Transformations
- Budget-Aware Pruning for Multi-Domain Learning
- Towards Recognizing New Semantic Concepts in New Visual Domains
- Multi-path Neural Networks for On-device Multi-domain Visual Classification
- What and Where: Learn to Plug Adapters via NAS for Multi-Domain Learning
- Parameter-efficient Multi-Task and Multi-Domain Learning using Factorized Tensor Networks
- Disentangling Transfer and Interference in Multi-Domain Learning