Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts

Abstract

Detecting objects becomes difficult when we need to deal with large shape deformation, occlusion and low resolution. We propose a novel approach to i) handle large deformations and partial occlusions in animals (as examples of highly deformable objects), ii) describe them in terms of body parts, and iii) detect them when their body parts are hard to detect (e.g., animals depicted at low resolution). We represent the holistic object and body parts separately and use a fully connected model to arrange templates for the holistic object and body parts. Our model automatically decouples the holistic object or body parts from the model when they are hard to detect. This enables us to represent a large number of holistic object and body part combinations to better deal with different "detectability" patterns caused by deformations, occlusion and/or low resolution. We apply our method to the six animal categories in the PASCAL VOC dataset and show that our method significantly improves state-of-the-art (by 4.1% AP) and provides a richer representation for objects. During training we use annotations for body parts (e.g., head, torso, etc), making use of a new dataset of fully annotated object parts for PASCAL VOC 2010, which provides a mask for each part.

CBMM memo

Cited by in corpus (54)

• A Review on Deep Learning Techniques Applied to Semantic Segmentation
• Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations
• Semantic Understanding of Scenes through the ADE20K Dataset
• Deeply-Learned Part-Aligned Representations for Person Re-Identification
• Light-Weight RefineNet for Real-Time Semantic Segmentation
• Attention-Aware Compositional Network for Person Re-identification
• RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
• Multiple-Human Parsing in the Wild
• segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
• Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
• Unite the People: Closing the Loop Between 3D and 2D Human Representations
• Deep Learning for Semantic Part Segmentation with High-Level Guidance
• Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
• Interpreting CNN Knowledge via an Explanatory Graph
• Semantic Object Parsing with Local-Global Long Short-Term Memory
• Joint Object and Part Segmentation using Deep Learned Potentials
• Learning Dense Correspondence via 3D-guided Cycle Consistency
• Instance-level Human Parsing via Part Grouping Network
• Interpretable Convolutional Neural Networks
• Hypercolumns for Object Segmentation and Fine-grained Localization
• DeePM: A Deep Part-Based Model for Object Detection and Semantic Part Localization
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• Interactively Transferring CNN Patterns for Part Localization
• Unified Perceptual Parsing for Scene Understanding
• Visual Concepts and Compositional Voting
• FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence
• Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net
• Cross-domain Human Parsing via Adversarial Feature and Label Adaptation
• Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
• Generative Partition Networks for Multi-Person Pose Estimation
• Joint Multi-Person Pose Estimation and Semantic Part Segmentation
• SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks
• Integrated perception with recurrent multi-task neural networks
• Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer
• Mining Object Parts from CNNs via Active Question-Answering
• Hierarchical semantic segmentation using modular convolutional neural networks
• Beyond One Glance: Gated Recurrent Architecture for Hand Segmentation
• Parsing Occluded People by Flexible Compositions
• Point and Ask: Incorporating Pointing into Visual Question Answering
• Keypoint Based Weakly Supervised Human Parsing
• Learning Semantic Neural Tree for Human Parsing
• DCTM: Discrete-Continuous Transformation Matching for Semantic Flow
• DeepVoting: A Robust and Explainable Deep Network for Semantic Part Detection under Partial Occlusion
• Adaptive Temporal Encoding Network for Video Instance-level Human Parsing
• EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use case
• Explanatory Graphs for CNNs
• Mining Interpretable AOG Representations from Convolutional Networks via Active Question Answering
• Deep, Dense, and Low-Rank Gaussian Conditional Random Fields
• 'Part'ly first among equals: Semantic part-based benchmarking for state-of-the-art object recognition systems
• Ground-truth dataset and baseline evaluations for image base-detail separation algorithms
• A CNN Cascade for Landmark Guided Semantic Part Segmentation
• Analyzing structural characteristics of object category representations from their semantic-part distributions
• Single Image Action Recognition using Semantic Body Part Actions
• Recognizing Car Fluents from Video