BoLTVOS: Box-Level Tracking for Video Object Segmentation
arXiv:1904.04552
Abstract
We approach video object segmentation (VOS) by splitting the task into two sub-tasks: bounding box level tracking, followed by bounding box segmentation. Following this paradigm, we present BoLTVOS (Box-Level Tracking for VOS), which consists of an R-CNN detector conditioned on the first-frame bounding box to detect the object of interest, a temporal consistency rescoring algorithm, and a Box2Seg network that converts bounding boxes to segmentation masks. BoLTVOS performs VOS using only the firstframe bounding box without the mask. We evaluate our approach on DAVIS 2017 and YouTube-VOS, and show that it outperforms all methods that do not perform first-frame fine-tuning. We further present BoLTVOS-ft, which learns to segment the object in question using the first-frame mask while it is being tracked, without increasing the runtime. BoLTVOS-ft outperforms PReMVOS, the previously best performing VOS method on DAVIS 2016 and YouTube-VOS, while running up to 45 times faster. Our bounding box tracker also outperforms all previous short-term and longterm trackers on the bounding box level tracking datasets OTB 2015 and LTB35. A newer version of this work can be found at arXiv:1911.12836.
References in corpus (6)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
- Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
- A Novel Performance Evaluation Methodology for Single-Target Trackers
- Vision Meets Drones: A Challenge
- Rethinking ImageNet Pre-training
Cited by in corpus (8)
- Collaborative Video Object Segmentation by Foreground-Background Integration
- Associating Objects with Transformers for Video Object Segmentation
- Efficient Regional Memory Network for Video Object Segmentation
- Fast Video Object Segmentation using the Global Context Module
- Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integration
- An Exploration of Target-Conditioned Segmentation Methods for Visual Object Trackers
- Fast Template Matching and Update for Video Object Tracking and Segmentation
- FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation