Instance and Panoptic Segmentation Using Conditional Convolutions

Abstract

We propose a simple yet effective framework for instance and panoptic segmentation, termed CondInst (conditional convolutions for instance and panoptic segmentation). In the literature, top-performing instance segmentation methods typically follow the paradigm of Mask R-CNN and rely on ROI operations (typically ROIAlign) to attend to each instance. In contrast, we propose to attend to the instances with dynamic conditional convolutions. Instead of using instance-wise ROIs as inputs to the instance mask head of fixed weights, we design dynamic instance-aware mask heads, conditioned on the instances to be predicted. CondInst enjoys three advantages: 1.) Instance and panoptic segmentation are unified into a fully convolutional network, eliminating the need for ROI cropping and feature alignment. 2.) The elimination of the ROI cropping also significantly improves the output instance mask resolution. 3.) Due to the much improved capacity of dynamically-generated conditional convolutions, the mask head can be very compact (e.g., 3 conv. layers, each having only 8 channels), leading to significantly faster inference time per instance and making the overall inference time almost constant, irrelevant to the number of instances. We demonstrate a simpler method that can achieve improved accuracy and inference speed on both instance and panoptic segmentation tasks. On the COCO dataset, we outperform a few state-of-the-art methods. We hope that CondInst can be a strong baseline for instance and panoptic segmentation. Code is available at: https://git.io/AdelaiDet

arXiv commentAccepted to IEEE T. Pattern Analysis and Machine Intelligence (TPAMI). Extended version of arXiv:2003.05664

References in corpus (21)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Focal Loss for Dense Object Detection
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• FCOS: Fully Convolutional One-Stage Object Detection
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• SOLOv2: Dynamic and Fast Instance Segmentation
• Path Aggregation Network for Instance Segmentation
• CondConv: Conditionally Parameterized Convolutions for Efficient Inference
• Semantic Instance Segmentation via Deep Metric Learning
• Hybrid Task Cascade for Instance Segmentation
• DeeperLab: Single-Shot Image Parser
• Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video
• CrowdNet: A Deep Convolutional Network for Dense Crowd Counting
• Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
• Panoptic Feature Pyramid Networks
• Instance-sensitive Fully Convolutional Networks
• Enforcing geometric constraints of virtual normal for depth prediction
• TensorMask: A Foundation for Dense Object Segmentation
• DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
• Conditional Convolutions for Instance Segmentation
• Unifying Training and Inference for Panoptic Segmentation

Cited by in corpus (2)

• SE-PSNet: Silhouette-based Enhancement Feature for Panoptic Segmentation Network
• Panoptic Segmentation: A Review