Weakly Supervised Object Detection in Artworks

Abstract

We propose a method for the weakly supervised detection of objects in paintings. At training time, only image-level annotations are needed. This, combined with the efficiency of our multiple-instance learning method, enables one to learn new classes on-the-fly from globally annotated databases, avoiding the tedious task of manually marking objects. We show on several databases that dropping the instance-level annotations only yields mild performance losses. We also introduce a new database, IconArt, on which we perform detection experiments on classes that could not be learned on photographs, such as Jesus Child or Saint Sebastian. To the best of our knowledge, these are the first experiments dealing with the automatic (and in our case weakly supervised) detection of iconographic elements in paintings. We believe that such a method is of great benefit for helping art historians to explore large digital databases.

arXiv commentAccepted at ECCV 2018 Workshop Computer Vision for Art Analysis - VISART 2018 14 pages, 5 figures

References in corpus (4)

• SSD: Single Shot MultiBox Detector
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Multiple Instance Learning: A Survey of Problem Characteristics and Applications
• Weakly Supervised Object Localization with Multi-fold Multiple Instance Learning

Cited by in corpus (4)

• Visual link retrieval and knowledge discovery in painting datasets
• A deep learning approach to clustering visual arts
• An analysis of the transfer learning of convolutional neural networks for artistic images
• Object Retrieval and Localization in Large Art Collections using Deep Multi-Style Feature Fusion and Iterative Voting