Generator evaluator-selector net for panoptic image segmentation and splitting unfamiliar objects into parts

Abstract

In machine learning and other fields, suggesting a good solution to a problem is usually a harder task than evaluating the quality of such a solution. This asymmetry is the basis for a large number of selection oriented methods that use a generator system to guess a set of solutions and an evaluator system to rank and select the best solutions. This work examines the use of this approach to the problem of panoptic image segmentation and class agnostic parts segmentation. The generator/evaluator approach for this case consists of two independent convolutional neural nets: a generator net that suggests variety segments corresponding to objects, stuff and parts regions in the image, and an evaluator net that chooses the best segments to be merged into the segmentation map. The result is a trial and error evolutionary approach in which a generator that guesses segments with low average accuracy, but with wide variability, can still produce good results when coupled with an accurate evaluator. The generator consists of a Pointer net that receives an image and a point in the image, and predicts the region of the segment containing the point. Generating and evaluating each segment separately is essential in this case since it demands exponentially fewer guesses compared to a system that guesses and evaluates the full segmentation map in each try. The classification of the selected segments is done by an independent region-specific classification net. This allows the segmentation to be class agnostic and hence, capable of segmenting unfamiliar categories that were not part of the training set. The method was examined on the COCO Panoptic segmentation benchmark and gave results comparable to those of the basic semantic segmentation and Mask-RCNN methods. In addition, the system was used for the task of splitting objects of unseen classes (that did not appear in the training set) into parts.

References in corpus (13)

• Weight Uncertainty in Neural Networks
• Recent Advances in Open Set Recognition: A Survey
• Fully Convolutional Networks for Semantic Segmentation
• Semantic Segmentation using Adversarial Networks
• Selective Classification for Deep Neural Networks
• Learning to Fuse Things and Stuff
• Evolving Deep Convolutional Neural Networks for Image Classification
• Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
• Panoptic Feature Pyramid Networks
• An End-to-End Network for Panoptic Segmentation
• UPSNet: A Unified Panoptic Segmentation Network
• Confidence from Invariance to Image Transformations
• Classifying a specific image region using convolutional nets with an ROI mask as input

Cited by in corpus (1)

• Panoptic Segmentation: A Review