Depthwise Convolution is All You Need for Learning Multiple Visual Domains

Abstract

There is a growing interest in designing models that can deal with images from different visual domains. If there exists a universal structure in different visual domains that can be captured via a common parameterization, then we can use a single model for all domains rather than one model per domain. A model aware of the relationships between different domains can also be trained to work on new domains with less resources. However, to identify the reusable structure in a model is not easy. In this paper, we propose a multi-domain learning architecture based on depthwise separable convolution. The proposed approach is based on the assumption that images from different domains share cross-channel correlations but have domain-specific spatial correlations. The proposed model is compact and has minimal overhead when being applied to new domains. Additionally, we introduce a gating mechanism to promote soft sharing between different domains. We evaluate our approach on Visual Decathlon Challenge, a benchmark for testing the ability of multi-domain models. The experiments show that our approach can achieve the highest score while only requiring 50% of the parameters compared with the state-of-the-art approaches.

References in corpus (9)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Overcoming catastrophic forgetting in neural networks
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Depthwise Separable Convolutions for Neural Machine Translation
• A Survey on Methods and Theories of Quantized Neural Networks
• Universal representations:The missing link between faces, text, planktons, and cat breeds
• Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
• VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation
• How Local is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization

Cited by in corpus (4)

• EXTD: Extremely Tiny Face Detector via Iterative Filter Reuse
• AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
• Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport
• EffCNet: An Efficient CondenseNet for Image Classification on NXP BlueBox