Algorithmic Ways of Seeing: Using Object Detection to Facilitate Art Exploration

Abstract

This Research through Design paper explores how object detection may be applied to a large digital art museum collection to facilitate new ways of encountering and experiencing art. We present the design and evaluation of an interactive application called SMKExplore, which allows users to explore a museum's digital collection of paintings by browsing through objects detected in the images, as a novel form of open-ended exploration. We provide three contributions. First, we show how an object detection pipeline can be integrated into a design process for visual exploration. Second, we present the design and development of an app that enables exploration of an art museum's collection. Third, we offer reflections on future possibilities for museums and HCI researchers to incorporate object detection techniques into the digitalization of museums.

References in corpus (8)

• Learning Transferable Visual Models From Natural Language Supervision
• One pixel attack for fooling deep neural networks
• Art and the science of generative AI: A deeper dive
• The Creativity of Text-to-Image Generation
• Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry
• Detecting People in Artwork with CNNs
• Sensitive Pictures: Emotional Interpretation in the Museum
• Labeling of Cultural Heritage Collections on the Intersection of Visual Analytics and Digital Humanities