Visual Saliency Based on Multiscale Deep Features

Abstract

Visual saliency is a fundamental problem in both cognitive and computational sciences, including computer vision. In this CVPR 2015 paper, we discover that a high-quality visual saliency model can be trained with multiscale features extracted using a popular deep learning architecture, convolutional neural networks (CNNs), which have had many successes in visual recognition tasks. For learning such saliency models, we introduce a neural network architecture, which has fully connected layers on top of CNNs responsible for extracting features at three different scales. We then propose a refinement method to enhance the spatial coherence of our saliency results. Finally, aggregating multiple saliency maps computed for different levels of image segmentation can further boost the performance, yielding saliency maps better than those generated from a single segmentation. To promote further research and evaluation of visual saliency models, we also construct a new large database of 4447 challenging images and their pixelwise saliency annotation. Experimental results demonstrate that our proposed method is capable of achieving state-of-the-art performance on all public benchmarks, improving the F-Measure by 5.0% and 13.2% respectively on the MSRA-B dataset and our new dataset (HKU-IS), and lowering the mean absolute error by 5.7% and 35.1% respectively on these two datasets.

To appear in CVPR 2015

References in corpus (3)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Rich feature hierarchies for accurate object detection and semantic segmentation

Cited by in corpus (39)

• Video Salient Object Detection via Fully Convolutional Networks
• RANet: Ranking Attention Network for Fast Video Object Segmentation
• DISC: Deep Image Saliency Computing via Progressive Representation Learning
• Deep Contrast Learning for Salient Object Detection
• Shallow and Deep Convolutional Networks for Saliency Prediction
• Edge Preserving and Multi-Scale Contextual Neural Network for Salient Object Detection
• Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection
• Structure-measure: A New Way to Evaluate Foreground Maps
• Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction
• Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
• PiCANet: Learning Pixel-wise Contextual Attention for Saliency Detection
• An Unsupervised Game-Theoretic Approach to Saliency Detection
• Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground
• Salient Object Detection in Video using Deep Non-Local Neural Networks
• Co-salient Object Detection Based on Deep Saliency Networks and Seed Propagation over an Integrated Graph
• Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey
• Learning Uncertain Convolutional Features for Accurate Saliency Detection
• Probabilistic Saliency Estimation
• Agile Amulet: Real-Time Salient Object Detection with Contextual Attention
• PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
• How do people explore virtual environments?
• Semantics Meet Saliency: Exploring Domain Affinity and Models for Dual-Task Prediction
• Saliency Detection by Forward and Backward Cues in Deep-CNNs
• AGIL: Learning Attention from Human for Visuomotor Tasks
• Deep Green Function Convolution for Improving Saliency in Convolutional Neural Networks
• HyperFusion-Net: Densely Reflective Fusion for Salient Object Detection
• Contrast-Oriented Deep Neural Networks for Salient Object Detection
• A Benchmark Dataset and Saliency-guided Stacked Autoencoders for Video-based Salient Object Detection
• Instance-Level Salient Object Segmentation
• Exploiting the Value of the Center-dark Channel Prior for Salient Object Detection
• DeepNav: Learning to Navigate Large Cities
• Quality-Aware Multimodal Saliency Detection via Deep Reinforcement Learning
• Deep, Dense, and Low-Rank Gaussian Conditional Random Fields
• Deep Saliency with Encoded Low level Distance Map and High Level Features
• Saliency Enhancement using Gradient Domain Edges Merging
• Graph-Theoretic Spatiotemporal Context Modeling for Video Saliency Detection
• Saliency Detection via Combining Region-Level and Pixel-Level Predictions with CNNs
• Visual Attention on the Sun: What Do Existing Models Actually Predict?
• Heterogeneous Grid Convolution for Adaptive, Efficient, and Controllable Computation