Class Subset Selection for Transfer Learning using Submodularity

Abstract

In recent years, it is common practice to extract fully-connected layer (fc) features that were learned while performing image classification on a source dataset, such as ImageNet, and apply them generally to a wide range of other tasks. The general usefulness of some large training datasets for transfer learning is not yet well understood, and raises a number of questions. For example, in the context of transfer learning, what is the role of a specific class in the source dataset, and how is the transferability of fc features affected when they are trained using various subsets of the set of all classes in the source dataset? In this paper, we address the question of how to select an optimal subset of the set of classes, subject to a budget constraint, that will more likely generate good features for other tasks. To accomplish this, we use a submodular set function to model the accuracy achievable on a new task when the features have been learned on a given subset of classes of the source dataset. An optimal subset is identified as the set that maximizes this submodular function. The maximization can be accomplished using an efficient greedy algorithm that comes with guarantees on the optimality of the solution. We empirically validate our submodular model by successfully identifying subsets of classes that produce good features for new tasks.

References in corpus (14)

• Intriguing properties of neural networks
• Deep Residual Learning for Image Recognition
• Understanding Deep Convolutional Networks
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Rich feature hierarchies for accurate object detection and semantic segmentation
• What makes ImageNet good for transfer learning?
• Unsupervised Visual Representation Learning by Context Prediction
• Unsupervised Learning of Visual Representations using Videos
• Learning to See by Moving
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• Understanding Intra-Class Knowledge Inside CNN
• Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
• Understanding deep features with computer-generated imagery
• Digging Deep into the layers of CNNs: In Search of How CNNs Achieve View Invariance