Hierarchical binary CNNs for landmark localization with limited resources

Abstract

Our goal is to design architectures that retain the groundbreaking performance of Convolutional Neural Networks (CNNs) for landmark localization and at the same time are lightweight, compact and suitable for applications with limited computational resources. To this end, we make the following contributions: (a) we are the first to study the effect of neural network binarization on localization tasks, namely human pose estimation and face alignment. We exhaustively evaluate various design choices, identify performance bottlenecks, and more importantly propose multiple orthogonal ways to boost performance. (b) Based on our analysis, we propose a novel hierarchical, parallel and multi-scale residual architecture that yields large performance improvement over the standard bottleneck block while having the same number of parameters, thus bridging the gap between the original network and its binarized counterpart. (c) We perform a large number of ablation studies that shed light on the properties and the performance of the proposed block. (d) We present results for experiments on the most challenging datasets for human pose estimation and face alignment, reporting in many cases state-of-the-art performance. (e) We further provide additional results for the problem of facial part segmentation. Code can be downloaded from https://www.adrianbulat.com/binary-cnn-landmark

Accepted to IEEE TPAMI18: Best of ICCV 2017 SI. Previously portions of this work appeared as arXiv:1703.00862, which was the conference version

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Fully Convolutional Networks for Semantic Segmentation
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources
• GoDP: Globally optimized dual pathway system for facial landmark localization in-the-wild

Cited by in corpus (7)

• Discrimination-aware Network Pruning for Deep Model Compression
• XNOR-Net++: Improved Binary Neural Networks
• Improved training of binary networks for human pose estimation and image recognition
• Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantization
• Facial age estimation by deep residual decision making
• AQD: Towards Accurate Fully-Quantized Object Detection
• Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks