Visual Saliency Detection Based on Multiscale Deep CNN Features

Abstract

Visual saliency is a fundamental problem in both cognitive and computational sciences, including computer vision. In this paper, we discover that a high-quality visual saliency model can be learned from multiscale features extracted using deep convolutional neural networks (CNNs), which have had many successes in visual recognition tasks. For learning such saliency models, we introduce a neural network architecture, which has fully connected layers on top of CNNs responsible for feature extraction at three different scales. The penultimate layer of our neural network has been confirmed to be a discriminative high-level feature vector for saliency detection, which we call deep contrast feature. To generate a more robust feature, we integrate handcrafted low-level features with our deep contrast feature. To promote further research and evaluation of visual saliency models, we also construct a new large database of 4447 challenging images and their pixelwise saliency annotations. Experimental results demonstrate that our proposed method is capable of achieving state-of-the-art performance on all public benchmarks, improving the F- measure by 6.12% and 10.0% respectively on the DUT-OMRON dataset and our new dataset (HKU-IS), and lowering the mean absolute error by 9% and 35.3% respectively on these two datasets.

Accepted for publication in IEEE Transactions on Image Processing

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Deep Contrast Learning for Salient Object Detection
• PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Edge-Preserving Coherence

Cited by in corpus (39)

• U${}^2$-Net: Going Deeper with Nested U-Structure for Salient Object Detection
• Joint Optic Disc and Cup Segmentation Based on Multi-label Deep Network and Polar Transformation
• Exploring Rich and Efficient Spatial Temporal Interactions for Real Time Video Salient Object Detection
• Object Detection with Deep Learning: A Review
• Cross-Modal Attentional Context Learning for RGB-D Object Detection
• Boundary-Aware Segmentation Network for Mobile and Web Applications
• Reverse Attention for Salient Object Detection
• Transformer Encoder with Multiscale Deep Learning for Pain Classification Using Physiological Signals
• REB: Reducing Biases in Representation for Industrial Anomaly Detection
• A Deep Spatial Contextual Long-term Recurrent Convolutional Network for Saliency Detection
• Selectivity or Invariance: Boundary-aware Salient Object Detection
• Multi-Scale Iterative Refinement Network for RGB-D Salient Object Detection
• Accurate RGB-D Salient Object Detection via Collaborative Learning
• Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining
• Three Birds One Stone: A General Architecture for Salient Object Segmentation, Edge Detection and Skeleton Extraction
• Structured Modeling of Joint Deep Feature and Prediction Refinement for Salient Object Detection
• Two-stream Collaborative Learning with Spatial-Temporal Attention for Video Classification
• Hyperspectral Remote Sensing Images Salient Object Detection: The First Benchmark Dataset and Baseline
• Non-Local Context Encoder: Robust Biomedical Image Segmentation against Adversarial Attacks
• Weakly Supervised Camouflaged Object Detection Based on the SAM Model and Mask Guidance
• Video Salient Object Detection via Adaptive Local-Global Refinement
• Facial Landmark Machines: A Backbone-Branches Architecture with Progressive Representation Learning
• Contrast-Oriented Deep Neural Networks for Salient Object Detection
• Relationship-Embedded Representation Learning for Grounding Referring Expressions
• A Plug-and-play Scheme to Adapt Image Saliency Deep Model for Video Data
• Multi-source weak supervision for saliency detection
• Edge Learning with Unmanned Ground Vehicle: Joint Path, Energy and Sample Size Planning
• Attention Driven Person Re-identification
• Relating Blindsight and AI: A Review
• Defocus Blur Detection via Salient Region Detection Prior
• Harvesting Visual Objects from Internet Images via Deep Learning Based Objectness Assessment
• Shape-Tailored Deep Neural Networks
• Visual Boundary Knowledge Translation for Foreground Segmentation
• SaLite : A light-weight model for salient object detection
• Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread
• Rethinking of the Image Salient Object Detection: Object-level Semantic Saliency Re-ranking First, Pixel-wise Saliency Refinement Latter
• Salient Object Detection with Purificatory Mechanism and Structural Similarity Loss
• Local Deep-Feature Alignment for Unsupervised Dimension Reduction
• ROSA: Robust Salient Object Detection against Adversarial Attacks