Full-Duplex Strategy for Video Object Segmentation

Abstract

Previous video object segmentation approaches mainly focus on using simplex solutions between appearance and motion, limiting feature collaboration efficiency among and across these two cues. In this work, we study a novel and efficient full-duplex strategy network (FSNet) to address this issue, by considering a better mutual restraint scheme between motion and appearance in exploiting the cross-modal features from the fusion and decoding stage. Specifically, we introduce the relational cross-attention module (RCAM) to achieve bidirectional message propagation across embedding sub-spaces. To improve the model's robustness and update the inconsistent features from the spatial-temporal embeddings, we adopt the bidirectional purification module (BPM) after the RCAM. Extensive experiments on five popular benchmarks show that our FSNet is robust to various challenging scenarios (e.g., motion blur, occlusion) and achieves favourable performance against existing cutting-edges both in the video object segmentation and video salient object detection tasks. The project is publicly available at: https://dpfan.net/FSNet.

arXiv commentAccepted at ICCV-2021 (Journal Submission). Project Page: http://dpfan.net/FSNet/

References in corpus (19)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Salient Object Detection: A Benchmark
• Video Salient Object Detection via Fully Convolutional Networks
• Salient Object Detection: A Survey
• RGB-D Salient Object Detection: A Survey
• Motion-Attentive Transition for Zero-Shot Video Object Segmentation
• FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos
• Progressively Normalized Self-Attention Network for Video Polyp Segmentation
• The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation
• Video Salient Object Detection Using Spatiotemporal Deep Features
• Learning Video Object Segmentation from Static Images
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
• Anchor Diffusion for Unsupervised Video Object Segmentation
• Click Carving: Segmenting Objects in Video with Point Clicks
• Motion-Appearance Interactive Encoding for Object Segmentation in Unconstrained Videos
• Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
• Making a Case for 3D Convolutions for Object Segmentation in Videos
• ScribbleBox: Interactive Annotation Framework for Video Object Segmentation

Cited by in corpus (2)

• Advances in Deep Concealed Scene Understanding
• Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation