A Deep Spatial Contextual Long-term Recurrent Convolutional Network for Saliency Detection

Abstract

Traditional saliency models usually adopt hand-crafted image features and human-designed mechanisms to calculate local or global contrast. In this paper, we propose a novel computational saliency model, i.e., deep spatial contextual long-term recurrent convolutional network (DSCLRCN) to predict where people looks in natural scenes. DSCLRCN first automatically learns saliency related local features on each image location in parallel. Then, in contrast with most other deep network based saliency models which infer saliency in local contexts, DSCLRCN can mimic the cortical lateral inhibition mechanisms in human visual system to incorporate global contexts to assess the saliency of each image location by leveraging the deep spatial long short-term memory (DSLSTM) model. Moreover, we also integrate scene context modulation in DSLSTM for saliency inference, leading to a novel deep spatial contextual LSTM (DSCLSTM) model. The whole network can be trained end-to-end and works efficiently when testing. Experimental results on two benchmark datasets show that DSCLRCN can achieve state-of-the-art performance on saliency detection. Furthermore, the proposed DSCLSTM model can significantly boost the saliency detection performance by incorporating both global spatial interconnections and scene context modulation, which may uncover novel inspirations for studies on them in computational saliency models.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ParseNet: Looking Wider to See Better
• Visual Saliency Detection Based on Multiscale Deep CNN Features
• ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
• Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation

Cited by in corpus (6)

• SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes
• What Catches the Eye? Visualizing and Understanding Deep Saliency Models
• A Dilated Inception Network for Visual Saliency Prediction
• DeepFeat: A Bottom Up and Top Down Saliency Model Based on Deep Features of Convolutional Neural Nets
• Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention
• Influence of Image Classification Accuracy on Saliency Map Estimation