Cost-Effective Training of Deep CNNs with Active Model Adaptation
arXiv:1802.05394
Abstract
Deep convolutional neural networks have achieved great success in various applications. However, training an effective DNN model for a specific task is rather challenging because it requires a prior knowledge or experience to design the network architecture, repeated trial-and-error process to tune the parameters, and a large set of labeled data to train the model. In this paper, we propose to overcome these challenges by actively adapting a pre-trained model to a new task with less labeled examples. Specifically, the pre-trained model is iteratively fine tuned based on the most useful examples. The examples are actively selected based on a novel criterion, which jointly estimates the potential contribution of an instance on optimizing the feature representation as well as improving the classification model for the target task. On one hand, the pre-trained model brings plentiful information from its original task, avoiding redesign of the network architecture or training from scratch; and on the other hand, the labeling cost can be significantly reduced by active label querying. Experiments on multiple datasets and different pre-trained models demonstrate that the proposed approach can achieve cost-effective training of DNNs.
arXiv comment9 pages
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Deep Residual Learning for Image Recognition
- Learning Transferable Features with Deep Adaptation Networks
- Deep Domain Confusion: Maximizing for Domain Invariance
- Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
- Deep Bayesian Active Learning with Image Data
- Partial Transfer Learning with Selective Adversarial Networks