RANet: Ranking Attention Network for Fast Video Object Segmentation

Abstract

Despite online learning (OL) techniques have boosted the performance of semi-supervised video object segmentation (VOS) methods, the huge time costs of OL greatly restrict their practicality. Matching based and propagation based methods run at a faster speed by avoiding OL techniques. However, they are limited by sub-optimal accuracy, due to mismatching and drifting problems. In this paper, we develop a real-time yet very accurate Ranking Attention Network (RANet) for VOS. Specifically, to integrate the insights of matching based and propagation based methods, we employ an encoder-decoder framework to learn pixel-level similarity and segmentation in an end-to-end manner. To better utilize the similarity maps, we propose a novel ranking attention module, which automatically ranks and selects these maps for fine-grained VOS performance. Experiments on DAVIS-16 and DAVIS-17 datasets show that our RANet achieves the best speed-accuracy trade-off, e.g., with 33 milliseconds per frame and J&F=85.5% on DAVIS-16. With OL, our RANet reaches J&F=87.1% on DAVIS-16, exceeding state-of-the-art VOS methods. The code can be found at https://github.com/Storife/RANet.

Accepted by ICCV 2019. 10 pages, 7 figures, 6 tables. The supplementary file can be found at https://csjunxu.github.io/paper/2019ICCV/RANet_supp.pdf ; Code is available at https://github.com/Storife/RANet

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
• Visual Saliency Based on Multiscale Deep Features
• The 2018 DAVIS Challenge on Video Object Segmentation
• Norm matters: efficient and accurate normalization schemes in deep networks
• Learning Video Object Segmentation from Static Images
• Selective Video Object Cutout
• Video Propagation Networks

Cited by in corpus (39)

• Bilateral Attention Network for RGB-D Salient Object Detection
• Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation
• Conditional Variational Image Deraining
• Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinement
• EGNet:Edge Guidance Network for Salient Object Detection
• Towards Accurate Pixel-wise Object Tracking by Attention Retrieval
• Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation
• Learning Video Object Segmentation from Unlabeled Videos
• Co-attention Propagation Network for Zero-Shot Video Object Segmentation
• TransVOS: Video Object Segmentation with Transformers
• Kernelized Memory Network for Video Object Segmentation
• Memory Aggregation Networks for Efficient Interactive Video Object Segmentation
• State-Aware Tracker for Real-Time Video Object Segmentation
• Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion
• Fast Video Object Segmentation using the Global Context Module
• Self-supervised Video Object Segmentation
• MAST: A Memory-Augmented Self-supervised Tracker
• SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolution
• TTVOS: Lightweight Video Object Segmentation with Adaptive Template Attention Module and Temporal Consistency Loss
• MUNet: Motion Uncertainty-aware Semi-supervised Video Object Segmentation
• CRVOS: Clue Refining Network for Video Object Segmentation
• LSMVOS: Long-Short-Term Similarity Matching for Video Object
• PMVOS: Pixel-Level Matching-Based Video Object Segmentation
• Fast Video Object Segmentation With Temporal Aggregation Network and Dynamic Template Matching
• Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks
• SwiftNet: Real-time Video Object Segmentation
• Dual Temporal Memory Network for Efficient Video Object Segmentation
• Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation
• Semi-Supervised Self-Growing Generative Adversarial Networks for Image Recognition
• Perceptual Consistency in Video Segmentation
• Video Instance Segmentation by Instance Flow Assembly
• Fast Pixel-Matching for Video Object Segmentation
• VideoClick: Video Object Segmentation with a Single Click
• Weakly Supervised Video Salient Object Detection
• Staged Depthwise Correlation and Feature Fusion for Siamese Object Tracking
• Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm
• Coherent Loss: A Generic Framework for Stable Video Segmentation
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• DyStaB: Unsupervised Object Segmentation via Dynamic-Static Bootstrapping