Full-Duplex Strategy for Video Object Segmentation

Abstract

Previous video object segmentation approaches mainly focus on using simplex solutions between appearance and motion, limiting feature collaboration efficiency among and across these two cues. In this work, we study a novel and efficient full-duplex strategy network (FSNet) to address this issue, by considering a better mutual restraint scheme between motion and appearance in exploiting the cross-modal features from the fusion and decoding stage. Specifically, we introduce the relational cross-attention module (RCAM) to achieve bidirectional message propagation across embedding sub-spaces. To improve the model's robustness and update the inconsistent features from the spatial-temporal embeddings, we adopt the bidirectional purification module (BPM) after the RCAM. Extensive experiments on five popular benchmarks show that our FSNet is robust to various challenging scenarios (e.g., motion blur, occlusion) and achieves favourable performance against existing cutting-edges both in the video object segmentation and video salient object detection tasks. The project is publicly available at: https://dpfan.net/FSNet.

Accepted at ICCV-2021 (Journal Submission). Project Page: http://dpfan.net/FSNet/

References in corpus (8)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Progressively Normalized Self-Attention Network for Video Polyp Segmentation
• The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation
• Anchor Diffusion for Unsupervised Video Object Segmentation
• Click Carving: Segmenting Objects in Video with Point Clicks
• Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
• ScribbleBox: Interactive Annotation Framework for Video Object Segmentation

Cited by in corpus (2)

• Advances in Deep Concealed Scene Understanding
• Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation