MultiGrain: a unified image embedding for classes and instances

Abstract

MultiGrain is a network architecture producing compact vector representations that are suited both for image classification and particular object retrieval. It builds on a standard classification trunk. The top of the network produces an embedding containing coarse and fine-grained information, so that images can be recognized based on the object class, particular object, or if they are distorted copies. Our joint training is simple: we minimize a cross-entropy loss for classification and a ranking loss that determines if two images are identical up to data augmentation, with no need for additional labels. A key component of MultiGrain is a pooling layer that takes advantage of high-resolution images with a network trained at a lower resolution. When fed to a linear classifier, the learned embeddings provide state-of-the-art classification accuracy. For instance, we obtain 79.4% top-1 accuracy with a ResNet-50 learned on Imagenet, which is a +1.8% absolute improvement over the AutoAugment method. When compared with the cosine similarity, the same embeddings perform on par with the state-of-the-art for image retrieval at moderate resolutions.

References in corpus (5)

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Densely Connected Convolutional Networks
• YFCC100M: The New Data in Multimedia Research
• AutoAugment: Learning Augmentation Policies from Data
• Augment your batch: better training with larger batches

Cited by in corpus (20)

• ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
• Early Convolutions Help Transformers See Better
• Focal Self-attention for Local-Global Interactions in Vision Transformers
• XCiT: Cross-Covariance Image Transformers
• LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
• A Battle of Network Structures: An Empirical Study of CNN, Transformer, and MLP
• ResNet strikes back: An improved training procedure in timm
• Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training
• Fixing the train-test resolution discrepancy
• MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark
• CAPE: Encoding Relative Positions with Continuous Augmented Positional Embeddings
• Data augmentation in Bayesian neural networks and the cold posterior effect
• The 2021 Image Similarity Dataset and Challenge
• Bag of Tricks and A Strong baseline for Image Copy Detection
• Vehicle Re-identification Method Based on Vehicle Attribute and Mutual Exclusion Between Cameras
• D${}^2$LV: A Data-Driven and Local-Verification Approach for Image Copy Detection
• Self-Distilled Self-Supervised Representation Learning
• A Benchmark on Tricks for Large-scale Image Retrieval
• Meta-learning of Pooling Layers for Character Recognition
• Ripple Attention for Visual Perception with Sub-quadratic Complexity