How do Convolutional Neural Networks Learn Design?

Abstract

In this paper, we aim to understand the design principles in book cover images which are carefully crafted by experts. Book covers are designed in a unique way, specific to genres which convey important information to their readers. By using Convolutional Neural Networks (CNN) to predict book genres from cover images, visual cues which distinguish genres can be highlighted and analyzed. In order to understand these visual clues contributing towards the decision of a genre, we present the application of Layer-wise Relevance Propagation (LRP) on the book cover image classification results. We use LRP to explain the pixel-wise contributions of book cover design and highlight the design elements contributing towards particular genres. In addition, with the use of state-of-the-art object and text detection methods, insights about genre-specific book cover designs are discovered.

arXiv commentAccepted by ICPR 2018

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
• Judging a Book By its Cover

Cited by in corpus (2)

• Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
• Towards Book Cover Design via Layout Graphs