Fixing the train-test resolution discrepancy: FixEfficientNet

Abstract

This paper provides an extensive analysis of the performance of the EfficientNet image classifiers with several recent training procedures, in particular one that corrects the discrepancy between train and test images. The resulting network, called FixEfficientNet, significantly outperforms the initial architecture with the same number of parameters. For instance, our FixEfficientNet-B0 trained without additional training data achieves 79.3% top-1 accuracy on ImageNet with 5.3M parameters. This is a +0.5% absolute improvement over the Noisy student EfficientNet-B0 trained with 300M unlabeled images. An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88.5% top-1 accuracy (top-5: 98.7%), which establishes the new state of the art for ImageNet with a single crop. These improvements are thoroughly evaluated with cleaner protocols than the one usually employed for Imagenet, and particular we show that our improvement remains in the experimental setting of ImageNet-v2, that is less prone to overfitting, and with ImageNet Real Labels. In both cases we also establish the new state of the art.

References in corpus (5)

• Do ImageNet Classifiers Generalize to ImageNet?
• Billion-scale semi-supervised learning for image classification
• Self-training with Noisy Student improves ImageNet classification
• Are we done with ImageNet?
• Adversarial Examples Improve Image Recognition

Cited by in corpus (26)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• EfficientNetV2: Smaller Models and Faster Training
• XCiT: Cross-Covariance Image Transformers
• Self-supervised Pretraining of Visual Features in the Wild
• ResKD: Residual-Guided Knowledge Distillation
• Skill-driven Recommendations for Job Transition Pathways
• Explaining Bayesian Neural Networks
• Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
• Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
• Self-supervised Neural Architecture Search
• Rethinking FUN: Frequency-Domain Utilization Networks
• Backward-Compatible Prediction Updates: A Probabilistic Approach
• Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities
• Recognition-Aware Learned Image Compression
• Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training
• Event Selection and Background Rejection in Time Projection Chambers Using Convolutional Neural Networks and a Specific Application to the AdEPT Gamma-ray Polarimeter Mission
• Multi-modal anticipation of stochastic trajectories in a dynamic environment with Conditional Variational Autoencoders
• Estimating Galactic Distances From Images Using Self-supervised Representation Learning
• Pufferfish: Communication-efficient Models At No Extra Cost
• CNN Acceleration by Low-rank Approximation with Quantized Factors
• Estimating the Brittleness of AI: Safety Integrity Levels and the Need for Testing Out-Of-Distribution Performance
• Learnable Adaptive Cosine Estimator (LACE) for Image Classification
• ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition
• State-of-the-art Techniques in Deep Edge Intelligence
• ESPN: Extremely Sparse Pruned Networks
• Data Augmentation via Structured Adversarial Perturbations