Deformable Part Models are Convolutional Neural Networks

Abstract

Deformable part models (DPMs) and convolutional neural networks (CNNs) are two widely used tools for visual recognition. They are typically viewed as distinct approaches: DPMs are graphical models (Markov random fields), while CNNs are "black-box" non-linear classifiers. In this paper, we show that a DPM can be formulated as a CNN, thus providing a novel synthesis of the two ideas. Our construction involves unrolling the DPM inference algorithm and mapping each step to an equivalent (and at times novel) CNN layer. From this perspective, it becomes natural to replace the standard image features used in DPM with a learned feature extractor. We call the resulting model DeepPyramid DPM and experimentally validate it on PASCAL VOC. DeepPyramid DPM significantly outperforms DPMs based on histograms of oriented gradients features (HOG) and slightly outperforms a comparable version of the recently introduced R-CNN detection system, while running an order of magnitude faster.

References in corpus (3)

• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• DenseNet: Implementing Efficient ConvNet Descriptor Pyramids
• Rich feature hierarchies for accurate object detection and semantic segmentation

Cited by in corpus (19)

• Deformable Convolutional Networks
• DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
• Playing Doom with SLAM-Augmented Deep Reinforcement Learning
• Untangling Local and Global Deformations in Deep Convolutional Networks for Image Classification and Sliding Window Detection
• Joint Object and Part Segmentation using Deep Learned Potentials
• Viewpoints and Keypoints
• Visual Representations: Defining Properties and Deep Approximations
• Crafting GBD-Net for Object Detection
• Factors of Transferability for a Generic ConvNet Representation
• Unsupervised Person Re-Identification: A Systematic Survey of Challenges and Solutions
• Actions and Attributes from Wholes and Parts
• When Kernel Methods meet Feature Learning: Log-Covariance Network for Action Recognition from Skeletal Data
• Weakly Supervised Object Localization Using Things and Stuff Transfer
• Structured Feature Learning for Pose Estimation
• ViP-CNN: Visual Phrase Guided Convolutional Neural Network
• End-to-End Integration of a Convolutional Network, Deformable Parts Model and Non-Maximum Suppression
• A Simple Hierarchical Pooling Data Structure for Loop Closure
• Convolutional Channel Features
• Detector-in-Detector: Multi-Level Analysis for Human-Parts