Scene Recognition with Prototype-agnostic Scene Layout

Abstract

Abstract--- Exploiting the spatial structure in scene images is a key research direction for scene recognition. Due to the large intra-class structural diversity, building and modeling flexible structural layout to adapt various image characteristics is a challenge. Existing structural modeling methods in scene recognition either focus on predefined grids or rely on learned prototypes, which all have limited representative ability. In this paper, we propose Prototype-agnostic Scene Layout (PaSL) construction method to build the spatial structure for each image without conforming to any prototype. Our PaSL can flexibly capture the diverse spatial characteristic of scene images and have considerable generalization capability. Given a PaSL, we build Layout Graph Network (LGN) where regions in PaSL are defined as nodes and two kinds of independent relations between regions are encoded as edges. The LGN aims to incorporate two topological structures (formed in spatial and semantic similarity dimensions) into image representations through graph convolution. Extensive experiments show that our approach achieves state-of-the-art results on widely recognized MIT67 and SUN397 datasets without multi-model or multi-scale fusion. Moreover, we also conduct the experiments on one of the largest scale datasets, Places365. The results demonstrate the proposed method can be well generalized and obtains competitive performance.

References in corpus (4)

• Object Detectors Emerge in Deep Scene CNNs
• Scene recognition with CNNs: objects, scales and dataset bias
• Semantic-Aware Scene Recognition
• Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image Retrieval

Cited by in corpus (10)

• Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction
• Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos
• 2D-3D Geometric Fusion Network using Multi-Neighbourhood Graph Convolution for RGB-D Indoor Scene Classification
• Deep Learning for Scene Classification: A Survey
• Inter-object Discriminative Graph Modeling for Indoor Scene Recognition
• Recent Advances in Scene Image Representation and Classification
• Leveraging Self-Supervised Learning for Scene Classification in Child Sexual Abuse Imagery
• DP-Net: Learning Discriminative Parts for image recognition
• Semantic-guided modeling of spatial relation and object co-occurrence for indoor scene recognition
• Reducing Redundancy in Data Organization and Arithmetic Calculation for Stencil Computations