Tag Prediction at Flickr: a View from the Darkroom
arXiv:1612.01922
Abstract
Automated photo tagging has established itself as one of the most compelling applications of deep learning. While deep convolutional neural networks have repeatedly demonstrated top performance on standard datasets for classification, there are a number of often overlooked but important considerations when deploying this technology in a real-world scenario. In this paper, we present our efforts in developing a large-scale photo tagging system for Flickr photo search. We discuss topics including how to 1) select the tags that matter most to our users; 2) develop lightweight, high-performance models for tag prediction; and 3) leverage the power of large amounts of noisy data for training. Our results demonstrate that, for real-world datasets, training exclusively with this noisy data yields performance on par with the standard paradigm of first pre-training on clean data and then fine-tuning. In addition, we observe that the models trained with user-generated data can yield better fine-tuning results when a small amount of clean data is available. As such, we advocate for the approach of harnessing user-generated data in large-scale systems.
arXiv commentPresented at the ACM Multimedia Thematic Workshops, 2017
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- YFCC100M: The New Data in Multimedia Research
- Learning to Segment Object Candidates
- Rethinking the Inception Architecture for Computer Vision
- Training Convolutional Networks with Noisy Labels
- Deep Convolutional Ranking for Multilabel Image Annotation
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
- Learning From Noisy Large-Scale Datasets With Minimal Supervision