Hybrid Task Cascade for Instance Segmentation

Abstract

Cascade is a classic yet powerful architecture that has boosted performance on various tasks. However, how to introduce cascade to instance segmentation remains an open question. A simple combination of Cascade R-CNN and Mask R-CNN only brings limited gain. In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation. In this work, we propose a new framework, Hybrid Task Cascade (HTC), which differs in two important aspects: (1) instead of performing cascaded refinement on these two tasks separately, it interweaves them for a joint multi-stage processing; (2) it adopts a fully convolutional branch to provide spatial context, which can help distinguishing hard foreground from cluttered background. Overall, this framework can learn more discriminative features progressively while integrating complementary features together in each stage. Without bells and whistles, a single HTC obtains 38.4 and 1.5 improvement over a strong Cascade Mask R-CNN baseline on MSCOCO dataset. Moreover, our overall system achieves 48.6 mask AP on the test-challenge split, ranking 1st in the COCO 2018 Challenge Object Detection Task. Code is available at: https://github.com/open-mmlab/mmdetection.

CVPR 2019 camera ready

References in corpus (6)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Learning to Segment Object Candidates
• Bridging Category-level and Instance-level Semantic Image Segmentation
• COCO-Stuff: Thing and Stuff Classes in Context
• Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs

Cited by in corpus (67)

• Deep High-Resolution Representation Learning for Visual Recognition
• Deep Learning for Generic Object Detection: A Survey
• Fast Camouflaged Object Detection via Edge-based Reversible Re-calibration Network
• K-Net: Towards Unified Image Segmentation
• Probabilistic two-stage detection
• R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object
• Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression
• Libra R-CNN: Towards Balanced Learning for Object Detection
• Masked-attention Mask Transformer for Universal Image Segmentation
• Cascade R-CNN: High Quality Object Detection and Instance Segmentation
• Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection
• Review of data analysis in vision inspection of power lines with an in-depth discussion of deep learning technology
• Light Field Salient Object Detection: A Review and Benchmark
• Arbitrary-Oriented Ship Detection through Center-Head Point Extraction
• Learning Modulated Loss for Rotated Object Detection
• Prime Sample Attention in Object Detection
• SOLO: Segmenting Objects by Locations
• PF-Net: Point Fractal Network for 3D Point Cloud Completion
• TensorMask: A Foundation for Dense Object Segmentation
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation
• Camouflaged Object Segmentation with Distraction Mining
• CentripetalNet: Pursuing High-quality Keypoint Pairs for Object Detection
• SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation
• Simple Training Strategies and Model Scaling for Object Detection
• Context-aware Cross-level Fusion Network for Camouflaged Object Detection
• Improving Camouflaged Object Detection with the Uncertainty of Pseudo-edge Labels
• Side-Aware Boundary Localization for More Precise Object Detection
• Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation
• Align Deep Features for Oriented Object Detection
• C-RPNs: Promoting Object Detection in real world via a Cascade Structure of Region Proposal Networks
• Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search
• MovieNet: A Holistic Dataset for Movie Understanding
• PolyTransform: Deep Polygon Transformer for Instance Segmentation
• 1st Place Solution of LVIS Challenge 2020: A Good Box is not a Guarantee of a Good Mask
• BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation
• SpatialFlow: Bridging All Tasks for Panoptic Segmentation
• Multi-Stage HRNet: Multiple Stage High-Resolution Network for Human Pose Estimation
• Rethinking Classification and Localization for Cascade R-CNN
• BoundarySqueeze: Image Segmentation as Boundary Squeezing
• Cascaded Human-Object Interaction Recognition
• Rethinking Training from Scratch for Object Detection
• Amodal Segmentation Based on Visible Region Segmentation and Shape Prior
• Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
• 2nd Place Solution for Waymo Open Dataset Challenge -- 2D Object Detection
• Seesaw Loss for Long-Tailed Instance Segmentation
• Recent Advances in Fibrosis and Scar Segmentation from Cardiac MRI: A State-of-the-Art Review and Future Perspectives
• Object-Guided Instance Segmentation for Biological Images
• Instance and Panoptic Segmentation Using Conditional Convolutions
• The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation
• Crossover Learning for Fast Online Video Instance Segmentation
• Detecting Reflections by Combining Semantic and Instance Segmentation
• Decoupled IoU Regression for Object Detection
• The MIS Check-Dam Dataset for Object Detection and Instance Segmentation Tasks
• AABO: Adaptive Anchor Box Optimization for Object Detection via Bayesian Sub-sampling
• Progressive Hard-case Mining across Pyramid Levels for Object Detection
• Probabilistic Ranking-Aware Ensembles for Enhanced Object Detections
• Renovating Parsing R-CNN for Accurate Multiple Human Parsing
• PointINS: Point-based Instance Segmentation
• Enforcing Reasoning in Visual Commonsense Reasoning
• JAS-GAN: Generative Adversarial Network Based Joint Atrium and Scar Segmentations on Unbalanced Atrial Targets
• Improving Aerial Instance Segmentation in the Dark with Self-Supervised Low Light Enhancement
• Neural Mesh Refiner for 6-DoF Pose Estimation
• Instance Segmentation Challenge Track Technical Report, VIPriors Workshop at ICCV 2021: Task-Specific Copy-Paste Data Augmentation Method for Instance Segmentation
• Towards Fine-grained Large Object Segmentation 1st Place Solution to 3D AI Challenge 2020 -- Instance Segmentation Track
• Recursively Refined R-CNN: Instance Segmentation with Self-RoI Rebalancing
• SE-PSNet: Silhouette-based Enhancement Feature for Panoptic Segmentation Network
• Synthesizing Photorealistic Images with Deep Generative Learning