Elucidating image-to-set prediction: An analysis of models, losses and datasets
arXiv:1904.05709
Abstract
In this paper, we identify an important reproducibility challenge in the image-to-set prediction literature that impedes proper comparisons among published methods, namely, researchers use different evaluation protocols to assess their contributions. To alleviate this issue, we introduce an image-to-set prediction benchmark suite built on top of five public datasets of increasing task complexity that are suitable for multi-label classification (VOC, COCO, NUS-WIDE, ADE20k and Recipe1M). Using the benchmark, we provide an in-depth analysis where we study the key components of current models, namely the choice of the image representation backbone as well as the set predictor design. Our results show that (1) exploiting better image representation backbones leads to higher performance boosts than enhancing set predictors, and (2) modeling both the label co-occurrences and ordering has a slight positive impact in terms of performance, whereas explicit cardinality prediction only helps when training on complex datasets, such as Recipe1M. To facilitate future image-to-set prediction research, we make the code, best models and dataset splits publicly available at: https://github.com/facebookresearch/image-to-set.
References in corpus (9)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- CNN: Single-label to Multi-label
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Multi-Label Prediction via Compressed Sensing
- Learning Deep Latent Spaces for Multi-Label Classification
- Order-Free RNN with Visual Attention for Multi-Label Classification
- Loss Functions for Multiset Prediction
- Joint Learning of Set Cardinality and State Distribution
- Structured Label Inference for Visual Understanding