Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature

Abstract

In the past few years, the number of fine-art collections that are digitized and publicly available has been growing rapidly. With the availability of such large collections of digitized artworks comes the need to develop multimedia systems to archive and retrieve this pool of data. Measuring the visual similarity between artistic items is an essential step for such multimedia systems, which can benefit more high-level multimedia tasks. In order to model this similarity between paintings, we should extract the appropriate visual features for paintings and find out the best approach to learn the similarity metric based on these features. We investigate a comprehensive list of visual features and metric learning approaches to learn an optimized similarity measure between paintings. We develop a machine that is able to make aesthetic-related semantic-level judgments, such as predicting a painting's style, genre, and artist, as well as providing similarity measures optimized based on the knowledge available in the domain of art historical interpretation. Our experiments show the value of using this similarity measure for the aforementioned prediction tasks.

21 pages

Cited by in corpus (10)

• Compacting, Picking and Growing for Unforgetting Continual Learning
• Context-Aware Embeddings for Automatic Art Analysis
• OmniArt: Multi-task Deep Learning for Artistic Data Analysis
• Boosting Binary Masks for Multi-Domain Learning through Affine Transformations
• Historical and Modern Features for Buddha Statue Classification
• KSM: Fast Multiple Task Adaption via Kernel-wise Soft Mask Learning
• Towards Recognizing New Semantic Concepts in New Visual Domains
• Unified Style Transfer
• Non-Parametric Neural Style Transfer
• CAMS: Color-Aware Multi-Style Transfer