Oriented Boxes for Accurate Instance Segmentation

Abstract

State-of-the-art instance-aware semantic segmentation algorithms use axis-aligned bounding boxes as an intermediate processing step to infer the final instance mask output. This often leads to coarse and inaccurate mask proposals due to the following reasons: Axis-aligned boxes have a high background to foreground pixel-ratio, there is a strong variation of mask targets with respect to the underlying box, and neighboring instances frequently reach into the axis-aligned bounding box of the instance mask of interest. In this work, we overcome these problems by proposing to use oriented boxes as the basis to infer instance masks. We show that oriented instance segmentation improves the mask predictions, especially when objects are diagonally aligned, touching, or overlapping each other. We evaluate our model on the D2S and Screws datasets and show that we can significantly improve the mask accuracy by 10% and 12% mAP compared to instance segmentation using axis-aligned bounding boxes, respectively. On the newly introduced Pill Bags dataset we outperform the baseline using only 10% of the mask annotations.

arXiv commentv3 with appendix

References in corpus (9)

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Automatic Ship Detection of Remote Sensing Images from Google Earth in Complex Scenes Based on Multi-Scale Rotation Dense Feature Pyramid Networks
• Rich feature hierarchies for accurate object detection and semantic segmentation
• R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• Learning a Rotation Invariant Detector with Rotatable Bounding Box
• RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free
• ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors
• Amodal Instance Segmentation