DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection

Abstract

A key problem in salient object detection is how to effectively model the semantic properties of salient objects in a data-driven manner. In this paper, we propose a multi-task deep saliency model based on a fully convolutional neural network (FCNN) with global input (whole raw images) and global output (whole saliency maps). In principle, the proposed saliency model takes a data-driven strategy for encoding the underlying saliency prior information, and then sets up a multi-task learning scheme for exploring the intrinsic correlations between saliency detection and semantic image segmentation. Through collaborative feature learning from such two correlated tasks, the shared fully convolutional layers produce effective features for object perception. Moreover, it is capable of capturing the semantic information on salient objects across different levels using the fully convolutional layers, which investigate the feature-sharing properties of salient object detection with great feature redundancy reduction. Finally, we present a graph Laplacian regularized nonlinear regression model for saliency refinement. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches.

To appear in IEEE Transactions on Image Processing (TIP), Project Website: http://www.zhaoliming.net/research/deepsaliency

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Fully Convolutional Networks for Semantic Segmentation
• Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
• What is a salient object? A dataset and a baseline model for salient object detection

Cited by in corpus (85)

• Deeply supervised salient object detection with short connections
• Video Salient Object Detection via Fully Convolutional Networks
• Regularized Deep Networks in Intelligent Transportation Systems: A Taxonomy and a Case Study
• RGBD Salient Object Detection via Deep Fusion
• Forecasting day-ahead electricity prices in Europe: the importance of considering market integration
• Predicting Video Saliency with Object-to-Motion CNN and Two-layer Convolutional LSTM
• Edge Preserving and Multi-Scale Contextual Neural Network for Salient Object Detection
• Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection
• Cascaded Partial Decoder for Fast and Accurate Salient Object Detection
• Video Salient Object Detection Using Spatiotemporal Deep Features
• Object Detection with Deep Learning: A Review
• Cross-layer Feature Pyramid Network for Salient Object Detection
• SAC-Net: Spatial Attenuation Context for Salient Object Detection
• Sequential vessel segmentation via deep channel attention network
• SG-FCN: A Motion and Memory-Based Deep Learning Model for Video Saliency Detection
• Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation
• Salient Object Detection with Lossless Feature Reflection and Weighted Structural Loss
• Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
• Position-Aware Relation Learning for RGB-Thermal Salient Object Detection
• Saliency for Fine-grained Object Recognition in Domains with Scarce Training Data
• Reverse Attention for Salient Object Detection
• Improved Person Re-Identification Based on Saliency and Semantic Parsing with Deep Neural Network Models
• Salient Objects in Clutter
• GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector
• Multi-modal Face Pose Estimation with Multi-task Manifold Deep Learning
• Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground
• Boundary-guided Feature Aggregation Network for Salient Object Detection
• Identity and Posture Recognition in Smart Beds with Deep Multitask Learning
• Embedding Visual Hierarchy with Deep Networks for Large-Scale Visual Recognition
• Salient Object Detection in Video using Deep Non-Local Neural Networks
• Exploiting saliency for object segmentation from image level labels
• Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning
• Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency
• Target Detection and Segmentation in Circular-Scan Synthetic-Aperture-Sonar Images using Semi-Supervised Convolutional Encoder-Decoders
• Siamese Network for RGB-D Salient Object Detection and Beyond
• A review of Recent Techniques for Person Re-Identification
• Learning Uncertain Convolutional Features for Accurate Saliency Detection
• Agile Amulet: Real-Time Salient Object Detection with Contextual Attention
• Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
• DNA: Deeply-supervised Nonlinear Aggregation for Salient Object Detection
• Richer and Deeper Supervision Network for Salient Object Detection
• Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
• Accurate RGB-D Salient Object Detection via Collaborative Learning
• Data Augmentation for Deep Learning-based Radio Modulation Classification
• Non-rigid Object Tracking via Deep Multi-scale Spatial-temporal Discriminative Saliency Maps
• DeepUSPS: Deep Robust Unsupervised Saliency Prediction With Self-Supervision
• Highly Efficient Salient Object Detection with 100K Parameters
• Deep Reasoning with Multi-Scale Context for Salient Object Detection
• Context Proposals for Saliency Detection
• Three Birds One Stone: A General Architecture for Salient Object Segmentation, Edge Detection and Skeleton Extraction
• Self-explanatory Deep Salient Object Detection
• SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks
• Exploring Image Enhancement for Salient Object Detection in Low Light Images
• Edge-guided Non-local Fully Convolutional Network for Salient Object Detection
• DSAL-GAN: Denoising based Saliency Prediction with Generative Adversarial Networks
• Complete the Look: Scene-based Complementary Product Recommendation
• Video Saliency Detection by 3D Convolutional Neural Networks
• Learning to Transfer for Traffic Forecasting via Multi-task Learning
• Neural ODEs for Image Segmentation with Level Sets
• HyperFusion-Net: Densely Reflective Fusion for Salient Object Detection
• Contrast-Oriented Deep Neural Networks for Salient Object Detection
• RecSal : Deep Recursive Supervision for Visual Saliency Prediction
• OGNet: Salient Object Detection with Output-guided Attention Module
• Personalized Saliency and its Prediction
• A Unified Framework for Shot Type Classification Based on Subject Centric Lens
• Multi-source weak supervision for saliency detection
• Ro-SOS: Metric Expression Network (MEnet) for Robust Salient Object Segmentation
• Quality-Aware Multimodal Saliency Detection via Deep Reinforcement Learning
• Global and Local Sensitivity Guided Key Salient Object Re-augmentation for Video Saliency Detection
• Ultrafast Video Attention Prediction with Coupled Knowledge Distillation
• W-net: Simultaneous segmentation of multi-anatomical retinal structures using a multi-task deep neural network
• Complementary Segmentation of Primary Video Objects with Reversible Flows
• Model-guided Multi-path Knowledge Aggregation for Aerial Saliency Prediction
• Weakly-Supervised Saliency Detection via Salient Object Subitizing
• Recursive Multi-model Complementary Deep Fusion forRobust Salient Object Detection via Parallel Sub Networks
• MASON: A Model AgnoStic ObjectNess Framework
• Attention based visual analysis for fast grasp planning with multi-fingered robotic hand
• Salient Instance Segmentation via Subitizing and Clustering
• Context Aware Road-user Importance Estimation (iCARE)
• Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN
• Interpreting Deep Neural Networks with Relative Sectional Propagation by Analyzing Comparative Gradients and Hostile Activations
• Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations
• RGB-T Image Saliency Detection via Collaborative Graph Learning
• A Review of Computer Vision Methods in Network Security