Semantic Redundancies in Image-Classification Datasets: The 10% You Don't Need

Abstract

Large datasets have been crucial to the success of deep learning models in the recent years, which keep performing better as they are trained with more labelled data. While there have been sustained efforts to make these models more data-efficient, the potential benefit of understanding the data itself, is largely untapped. Specifically, focusing on object recognition tasks, we wonder if for common benchmark datasets we can do better than random subsets of the data and find a subset that can generalize on par with the full dataset when trained on. To our knowledge, this is the first result that can find notable redundancies in CIFAR-10 and ImageNet datasets (at least 10%). Interestingly, we observe semantic correlations between required and redundant images. We hope that our findings can motivate further research into identifying additional redundancies and exploiting them for more efficient training or data-collection.

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Scikit-learn: Machine Learning in Python
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Shake-Shake regularization
• Not All Samples Are Created Equal: Deep Learning with Importance Sampling
• Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
• Are All Training Examples Created Equal? An Empirical Study
• Learning From Less Data: Diversified Subset Selection and Active Learning in Image Classification Tasks

Cited by in corpus (8)

• Are all negatives created equal in contrastive instance discrimination?
• A critical look at the current train/test split in machine learning
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Understanding Isomorphism Bias in Graph Data Sets
• Sequential Graph Convolutional Network for Active Learning
• Using Wavelets and Spectral Methods to Study Patterns in Image-Classification Datasets
• Sampling Bias in Deep Active Classification: An Empirical Study
• Using Wavelets to Analyze Similarities in Image-Classification Datasets