Numerical Coordinate Regression with Convolutional Neural Networks

Abstract

We study deep learning approaches to inferring numerical coordinates for points of interest in an input image. Existing convolutional neural network-based solutions to this problem either take a heatmap matching approach or regress to coordinates with a fully connected output layer. Neither of these approaches is ideal, since the former is not entirely differentiable, and the latter lacks inherent spatial generalization. We propose our differentiable spatial to numerical transform (DSNT) to fill this gap. The DSNT layer adds no trainable parameters, is fully differentiable, and exhibits good spatial generalization. Unlike heatmap matching, DSNT works well with low heatmap resolutions, so it can be dropped in as an output layer for a wide range of existing fully convolutional architectures. Consequently, DSNT offers a better trade-off between inference speed and prediction accuracy compared to existing techniques. When used to replace the popular heatmap matching approach used in almost all state-of-the-art methods for pose estimation, DSNT gives better prediction accuracy for all model architectures tested.

References in corpus (9)

• Human pose estimation via Convolutional Part Heatmap Regression
• Stacked Hourglass Networks for Human Pose Estimation
• Convolutional Pose Machines
• Human Pose Regression by Combining Indirect Part Detection and Contextual Information
• Multi-Context Attention for Human Pose Estimation
• LIFT: Learned Invariant Feature Transform
• Learning Feature Pyramids for Human Pose Estimation
• Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation
• Self Adversarial Training for Human Pose Estimation

Cited by in corpus (28)

• Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods
• EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking
• Deep Learning-Based Human Pose Estimation: A Survey
• TFPose: Direct Human Pose Estimation with Transformers
• How Robust is 3D Human Pose Estimation to Occlusion?
• Heatmap Regression via Randomized Rounding
• Synthetic Occlusion Augmentation with Volumetric Heatmaps for the 2018 ECCV PoseTrack Challenge on 3D Human Pose Estimation
• Pixel-wise Regression: 3D Hand Pose Estimation via Spatial-form Representation and Differentiable Decoder
• Fast robust peg-in-hole insertion with continuous visual servoing
• Integral Human Pose Regression
• LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood
• Residual Block-based Multi-Label Classification and Localization Network with Integral Regression for Vertebrae Labeling
• DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image
• landmarker: a Toolkit for Anatomical Landmark Localization in 2D/3D Images
• End-to-End Learnable Geometric Vision by Backpropagating PnP Optimization
• Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments
• Head and Tail Localization of C. elegans
• Ki-Pode: Keypoint-based Implicit Pose Distribution Estimation of Rigid Objects
• Single Person Pose Estimation: A Survey
• Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking
• Interpretable and Flexible Target-Conditioned Neural Planners For Autonomous Vehicles
• DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-scale Consistency
• Spine Landmark Localization with combining of Heatmap Regression and Direct Coordinate Regression
• Gaussian Vector: An Efficient Solution for Facial Landmark Detection
• Doppler Spectrum Classification with CNNs via Heatmap Location Encoding and a Multi-head Output Layer
• Sparse to Dense Motion Transfer for Face Image Animation
• Simultaneous Denoising and Localization Network for Photoacoustic Target Localization
• On Coordinate Decoding for Keypoint Estimation Tasks