Video Salient Object Detection Using Spatiotemporal Deep Features

Abstract

This paper presents a method for detecting salient objects in videos where temporal information in addition to spatial information is fully taken into account. Following recent reports on the advantage of deep features over conventional hand-crafted features, we propose a new set of SpatioTemporal Deep (STD) features that utilize local and global contexts over frames. We also propose new SpatioTemporal Conditional Random Field (STCRF) to compute saliency from STD features. STCRF is our extension of CRF to the temporal domain and describes the relationships among neighboring regions both in a frame and over frames. STCRF leads to temporally consistent saliency maps over frames, contributing to the accurate detection of salient objects' boundaries and noise reduction during detection. Our proposed method first segments an input video into multiple scales and then computes a saliency map at each scale level using STD features with STCRF. The final saliency map is computed by fusing saliency maps at different scale levels. Our experiments, using publicly available benchmark datasets, confirm that the proposed method significantly outperforms state-of-the-art methods. We also applied our saliency computation to the video object segmentation task, showing that our method outperforms existing video object segmentation methods.

accepted at TIP

References in corpus (2)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Video Salient Object Detection via Fully Convolutional Networks

Cited by in corpus (12)

• Anabranch Network for Camouflaged Object Segmentation
• Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images
• MirrorNet: Bio-Inspired Camouflaged Object Segmentation
• Advances in Deep Concealed Scene Understanding
• Exploring Rich and Efficient Spatial Temporal Interactions for Real Time Video Salient Object Detection
• Salient Object Detection in Video using Deep Non-Local Neural Networks
• Full-Duplex Strategy for Video Object Segmentation
• Motion Guided Attention for Video Salient Object Detection
• Unsupervised motion saliency map estimation based on optical flow inpainting
• A Plug-and-play Scheme to Adapt Image Saliency Deep Model for Video Data
• Trajectory saliency detection using consistency-oriented latent codes from a recurrent auto-encoder
• Class agnostic moving target detection by color and location prediction of moving area