Unified Perceptual Parsing for Scene Understanding

Abstract

Humans recognize the visual world at multiple levels: we effortlessly categorize scenes and detect objects inside, while also identifying the textures and surfaces of the objects along with their different compositional parts. In this paper, we study a new task called Unified Perceptual Parsing, which requires the machine vision systems to recognize as many visual concepts as possible from a given image. A multi-task framework called UPerNet and a training strategy are developed to learn from heterogeneous image annotations. We benchmark our framework on Unified Perceptual Parsing and show that it is able to effectively segment a wide range of concepts from images. The trained networks are further applied to discover visual knowledge in natural scenes. Models are available at \url{https://github.com/CSAILVision/unifiedparsing}.

arXiv commentAccepted to European Conference on Computer Vision (ECCV) 2018

References in corpus (8)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Learning Deconvolution Network for Semantic Segmentation
• Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
• Interpreting Deep Visual Representations via Network Dissection
• MegDet: A Large Mini-Batch Object Detector
• Learning to Segment Every Thing
• Convolutional Models for Joint Object Categorization and Pose Estimation

Cited by in corpus (2)

• TextMountain: Accurate Scene Text Detection via Instance Segmentation
• See More Than Once -- Kernel-Sharing Atrous Convolution for Semantic Segmentation