Learning to Refine Object Segments

Abstract

Object segmentation requires both object-level information and low-level pixel data. This presents a challenge for feedforward networks: lower layers in convolutional nets capture rich spatial information, while upper layers encode object-level knowledge but are invariant to factors such as pose and appearance. In this work we propose to augment feedforward nets for object segmentation with a novel top-down refinement approach. The resulting bottom-up/top-down architecture is capable of efficiently generating high-fidelity object masks. Similarly to skip connections, our approach leverages features at all layers of the net. Unlike skip connections, our approach does not attempt to output independent predictions at each layer. Instead, we first output a coarse `mask encoding' in a feedforward pass, then refine this mask encoding in a top-down pass utilizing features at successively lower layers. The approach is simple, fast, and effective. Building on the recent DeepMask network for generating object proposals, we show accuracy improvements of 10-20% in average recall for various setups. Additionally, by optimizing the overall network architecture, our approach, which we call SharpMask, is 50% faster than the original DeepMask network (under .8s per image).

extended version of ECCV camera-ready (figures 6-9 only in arXiv)

References in corpus (3)

• Learning Deconvolution Network for Semantic Segmentation
• Fully Connected Deep Structured Networks
• Simultaneous Detection and Segmentation

Cited by in corpus (15)

• Path Aggregation Network for Instance Segmentation
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network
• Gland Instance Segmentation Using Deep Multichannel Neural Networks
• Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter
• Real-time Semantic Image Segmentation via Spatial Sparsity
• Flood-Filling Networks
• All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation
• Deep Multiphase Level Set for Scene Parsing
• Instance Shadow Detection
• TW-SMNet: Deep Multitask Learning of Tele-Wide Stereo Matching
• Mixed context networks for semantic segmentation
• Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics
• Instance-Level Salient Object Segmentation
• Synthesizing Photorealistic Images with Deep Generative Learning