A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets

Abstract

The original ImageNet dataset is a popular large-scale benchmark for training Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm design, architecture search, and hyperparameter tuning) on the original dataset might be prohibitive, we propose to consider a downsampled version of ImageNet. In contrast to the CIFAR datasets and earlier downsampled versions of ImageNet, our proposed ImageNet32$\times$32 (and its variants ImageNet64$\times$64 and ImageNet16$\times$16) contains exactly the same number of classes and images as ImageNet, with the only difference that the images are downsampled to 32$\times$32 pixels per image (64$\times$64 and 16$\times$16 pixels for the variants, respectively). Experiments on these downsampled variants are dramatically faster than on the original ImageNet and the characteristics of the downsampled datasets with respect to optimal hyperparameters appear to remain similar. The proposed datasets and scripts to reproduce our results are available at http://image-net.org/download-images and https://github.com/PatrykChrabaszcz/Imagenet32_Scripts

References in corpus (4)

• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Systematic evaluation of CNN advances on the ImageNet
• Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Cited by in corpus (116)

• Neural Architecture Search: A Survey
• AutoML: A Survey of the State-of-the-Art
• A Survey on Evolutionary Neural Architecture Search
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning
• Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
• NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
• Scaling Laws for Autoregressive Generative Modeling
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
• FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
• LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
• Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
• NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
• Deep Feature Space: A Geometrical Perspective
• TBD: Benchmarking and Analyzing Deep Neural Network Training
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• Semantic Segmentation with Labeling Uncertainty and Class Imbalance
• Neural Architecture Search without Training
• Why is the Mahalanobis Distance Effective for Anomaly Detection?
• Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
• A framework for the extraction of Deep Neural Networks by leveraging public data
• PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks
• Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?
• Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
• Iterative Learning with Open-set Noisy Labels
• Evaluating Efficient Performance Estimators of Neural Architectures
• Randomized Smoothing of All Shapes and Sizes
• Lossless Image Compression through Super-Resolution
• Class-incremental Learning via Deep Model Consolidation
• Defining Benchmarks for Continual Few-Shot Learning
• FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes
• Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
• Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark
• Consensus Control for Decentralized Deep Learning
• Lossy Image Compression with Normalizing Flows
• Deep Neural Network Fingerprinting by Conferrable Adversarial Examples
• Model-based Asynchronous Hyperparameter and Neural Architecture Search
• GuidedMixup: An Efficient Mixup Strategy Guided by Saliency Maps
• STEERAGE: Synthesis of Neural Networks Using Architecture Search and Grow-and-Prune Methods
• Stackelberg GAN: Towards Provable Minimax Equilibrium via Multi-Generator Architectures
• Neural Ensemble Search for Uncertainty Estimation and Dataset Shift
• Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
• Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity
• Using a thousand optimization tasks to learn hyperparameter search strategies
• Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup
• Stronger NAS with Weaker Predictors
• Rethinking the backbone architecture for tiny object detection
• Network Architecture Search for Domain Adaptation
• DVOLVER: Efficient Pareto-Optimal Neural Network Architecture Search
• Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning
• Invertible DenseNets with Concatenated LipSwish
• Efficient end-to-end learning for quantizable representations
• ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
• Cyclic Differentiable Architecture Search
• Contrastive Learning Improves Model Robustness Under Label Noise
• Taxonomy of Saliency Metrics for Channel Pruning
• Depthwise Multiception Convolution for Reducing Network Parameters without Sacrificing Accuracy
• Invertible Generative Modeling using Linear Rational Splines
• Local and Global Context-and-Object-part-Aware Superpixel-based Data Augmentation for Deep Visual Recognition
• The Intrinsic Manifolds of Radiological Images and their Role in Deep Learning
• Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift
• An Introduction to Neural Architecture Search for Convolutional Networks
• Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training
• Robust Training with Ensemble Consensus
• Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
• Controlling Over-generalization and its Effect on Adversarial Examples Generation and Detection
• Local Unsupervised Learning for Image Analysis
• Instance Correction for Learning with Open-set Noisy Labels
• DrNAS: Dirichlet Neural Architecture Search
• Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics
• Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
• Representation Learning via Consistent Assignment of Views to Clusters
• Catch-Up Mix: Catch-Up Class for Struggling Filters in CNN
• Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
• Increasing Depth Leads to U-Shaped Test Risk in Over-parameterized Convolutional Networks
• Ensemble Soft-Margin Softmax Loss for Image Classification
• Energy-efficient Amortized Inference with Cascaded Deep Classifiers
• Trainless Model Performance Estimation for Neural Architecture Search
• Sample Selection Using Multi-Task Autoencoders in Federated Learning with Non-IID Data
• Compressing gradients by exploiting temporal correlation in momentum-SGD
• FEAR: A Simple Lightweight Method to Rank Architectures
• Learnable Uncertainty under Laplace Approximations
• Neural Architecture Search with Random Labels
• Being a Bit Frequentist Improves Bayesian Neural Networks
• Learning Non-linear Wavelet Transformation via Normalizing Flow
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
• The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods
• BORE: Bayesian Optimization by Density-Ratio Estimation
• Factorized Neural Processes for Neural Processes: $K$-Shot Prediction of Neural Responses
• Adversarial AutoMixup
• Confusable Learning for Large-class Few-Shot Classification
• OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression
• D-PCN: Parallel Convolutional Networks for Image Recognition via a Discriminator
• ExCon: Explanation-driven Supervised Contrastive Learning for Image Classification
• GenURL: A General Framework for Unsupervised Representation Learning
• Intra-Model Collaborative Learning of Neural Networks
• Attending Category Disentangled Global Context for Image Classification
• Adversarial Training with Stochastic Weight Average
• Skeptical Deep Learning with Distribution Correction
• Training CNNs faster with Dynamic Input and Kernel Downsampling
• Self-supervised Knowledge Distillation Using Singular Value Decomposition
• Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones?
• Human Annotations Improve GAN Performances
• Signed Input Regularization
• Analyzing the Dependency of ConvNets on Spatial Information
• End-to-End Efficient Representation Learning via Cascading Combinatorial Optimization
• Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator
• AmoebaContact and GDFold: a new pipeline for rapid prediction of protein structures
• Distilling Double Descent
• Graphs for deep learning representations
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay
• Lossless Image Compression Using a Multi-Scale Progressive Statistical Model
• Midpoint Regularization: from High Uncertainty Training to Conservative Classification