Video Region Annotation with Sparse Bounding Boxes

Abstract

Video analysis has been moving towards more detailed interpretation (e.g. segmentation) with encouraging progresses. These tasks, however, increasingly rely on densely annotated training data both in space and time. Since such annotation is labour-intensive, few densely annotated video data with detailed region boundaries exist. This work aims to resolve this dilemma by learning to automatically generate region boundaries for all frames of a video from sparsely annotated bounding boxes of target regions. We achieve this with a Volumetric Graph Convolutional Network (VGCN), which learns to iteratively find keypoints on the region boundaries using the spatio-temporal volume of surrounding appearance and motion. The global optimization of VGCN makes it significantly stronger and generalize better than existing solutions. Experimental results using two latest datasets (one real and one synthetic), including ablation studies, demonstrate the effectiveness and superiority of our method.

arXiv commentAccepted for publication in BMVC 2020 (Oral)

References in corpus (5)

• Unsupervised Learning of Visual Representations using Videos
• FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos
• Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals
• Future Semantic Segmentation with Convolutional LSTM
• Fast Interactive Object Annotation with Curve-GCN