Few-Shot Segmentation Propagation with Guided Networks

Abstract

Learning-based methods for visual segmentation have made progress on particular types of segmentation tasks, but are limited by the necessary supervision, the narrow definitions of fixed tasks, and the lack of control during inference for correcting errors. To remedy the rigidity and annotation burden of standard approaches, we address the problem of few-shot segmentation: given few image and few pixel supervision, segment any images accordingly. We propose guided networks, which extract a latent task representation from any amount of supervision, and optimize our architecture end-to-end for fast, accurate few-shot segmentation. Our method can switch tasks without further optimization and quickly update when given more guidance. We report the first results for segmentation from one pixel per concept and show real-time interactive video segmentation. Our unified approach propagates pixel annotations across space for interactive segmentation, across time for video segmentation, and across scenes for semantic segmentation. Our guided segmentor is state-of-the-art in accuracy for the amount of annotation and time. See http://github.com/shelhamer/revolver for code, models, and more details.

References in corpus (7)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• A Review on Deep Learning Techniques Applied to Semantic Segmentation
• Matching Networks for One Shot Learning
• Few-Shot Learning with Graph Neural Networks
• Learning feed-forward one-shot learners
• Deep Interactive Object Selection

Cited by in corpus (25)

• FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
• PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment
• One-Shot Instance Segmentation
• Data augmentation using learned transformations for one-shot medical image segmentation
• Hypercorrelation Squeeze for Few-Shot Segmentation
• MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation
• BriNet: Towards Bridging the Intra-class and Inter-class Gaps in One-Shot Segmentation
• Part-aware Prototype Network for Few-shot Semantic Segmentation
• Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation
• Few-Shot Viewpoint Estimation
• PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask
• Prototype-based Incremental Few-Shot Semantic Segmentation
• Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary
• A Novel BiLevel Paradigm for Image-to-Image Translation
• Attentional Prototype Inference for Few-Shot Segmentation
• Self-Supervised Tuning for Few-Shot Segmentation
• SML: Semantic Meta-learning for Few-shot Semantic Segmentation
• A Location-Sensitive Local Prototype Network for Few-Shot Medical Image Segmentation
• Multi-Stage Fusion for One-Click Segmentation
• Label Super Resolution with Inter-Instance Loss
• An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling
• Conditional Deep Convolutional Neural Networks for Improving the Automated Screening of Histopathological Images
• A Few-Shot Sequential Approach for Object Counting
• One-shot Weakly-Supervised Segmentation in Medical Images