DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Abstract

We introduce DiscoBox, a novel framework that jointly learns instance segmentation and semantic correspondence using bounding box supervision. Specifically, we propose a self-ensembling framework where instance segmentation and semantic correspondence are jointly guided by a structured teacher in addition to the bounding box supervision. The teacher is a structured energy model incorporating a pairwise potential and a cross-image potential to model the pairwise pixel relationships both within and across the boxes. Minimizing the teacher energy simultaneously yields refined object masks and dense correspondences between intra-class objects, which are taken as pseudo-labels to supervise the task network and provide positive/negative correspondence pairs for dense constrastive learning. We show a symbiotic relationship where the two tasks mutually benefit from each other. Our best model achieves 37.9% AP on COCO instance segmentation, surpassing prior weakly supervised methods and is competitive to supervised methods. We also obtain state of the art weakly supervised results on PASCAL VOC12 and PF-PASCAL with real-time inference.

Tech Report

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• YOLACT++: Better Real-time Instance Segmentation
• DenseBox: Unifying Landmark Localization with End to End Object Detection
• TILDE: A Temporally Invariant Learned DEtector
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• SPair-71k: A Large-scale Benchmark for Semantic Correspondence
• Conditional Convolutions for Instance Segmentation