Deep Aesthetic Quality Assessment with Semantic Information

Abstract

Human beings often assess the aesthetic quality of an image coupled with the identification of the image's semantic content. This paper addresses the correlation issue between automatic aesthetic quality assessment and semantic recognition. We cast the assessment problem as the main task among a multi-task deep model, and argue that semantic recognition task offers the key to address this problem. Based on convolutional neural networks, we employ a single and simple multi-task framework to efficiently utilize the supervision of aesthetic and semantic labels. A correlation item between these two tasks is further introduced to the framework by incorporating the inter-task relationship learning. This item not only provides some useful insight about the correlation but also improves assessment accuracy of the aesthetic task. Particularly, an effective strategy is developed to keep a balance between the two tasks, which facilitates to optimize the parameters of the framework. Extensive experiments on the challenging AVA dataset and Photo.net dataset validate the importance of semantic recognition in aesthetic quality assessment, and demonstrate that multi-task deep models can discover an effective aesthetic representation to achieve state-of-the-art results.

13 pages, 10 figures

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Fully Convolutional Networks for Semantic Segmentation
• Multi-task CNN Model for Attribute Prediction

Cited by in corpus (14)

• Stimuli-Aware Visual Emotion Analysis
• Composition and Style Attributes Guided Image Aesthetic Assessment
• A deep architecture for unified aesthetic prediction
• Theme-Aware Aesthetic Distribution Prediction With Full-Resolution Photographs
• Aesthetics Assessment of Images Containing Faces
• User-Guided Personalized Image Aesthetic Assessment based on Deep Reinforcement Learning
• Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon Divergence
• An End-to-End Neural Network for Image Cropping by Learning Composition from Aesthetic Photos
• A Hypothesis for the Aesthetic Appreciation in Neural Networks
• Composition-Aware Image Aesthetics Assessment
• Aesthetic Attributes Assessment of Images
• Image Aesthetics Prediction Using Multiple Patches Preserving the Original Aspect Ratio of Contents
• Tumera: Tutor of Photography Beginners
• A Gated Peripheral-Foveal Convolutional Neural Network for Unified Image Aesthetic Prediction