StarMap for Category-Agnostic Keypoint and Viewpoint Estimation
arXiv:1803.09331
Abstract
Semantic keypoints provide concise abstractions for a variety of visual understanding tasks. Existing methods define semantic keypoints separately for each category with a fixed number of semantic labels in fixed indices. As a result, this keypoint representation is in-feasible when objects have a varying number of parts, e.g. chairs with varying number of legs. We propose a category-agnostic keypoint representation, which combines a multi-peak heatmap (StarMap) for all the keypoints and their corresponding features as 3D locations in the canonical viewpoint (CanViewFeature) defined for each instance. Our intuition is that the 3D locations of the keypoints in canonical object views contain rich semantic and compositional information. Using our flexible representation, we demonstrate competitive performance in keypoint detection and localization compared to category-specific state-of-the-art methods. Moreover, we show that when augmented with an additional depth channel (DepthMap) to lift the 2D keypoints to 3D, our representation can achieve state-of-the-art results in viewpoint estimation. Finally, we show that our category-agnostic keypoint representation can be generalized to novel categories.
arXiv commentECCV 2018. Supplementary material with more qualitative results and higher resolution is available on the code page
References in corpus (20)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Deep Residual Learning for Image Recognition
- DeepPose: Human Pose Estimation via Deep Neural Networks
- VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
- Associative Embedding: End-to-End Learning for Joint Detection and Grouping
- Stacked Hourglass Networks for Human Pose Estimation
- Convolutional Pose Machines
- Pixels to Graphs by Associative Embedding
- Model-based Deep Hand Pose Estimation
- Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
- Learning Feature Pyramids for Human Pose Estimation
- 6-DoF Object Pose from Semantic Keypoints
- 3D Bounding Box Estimation Using Deep Learning and Geometry
- Dense Human Body Correspondences Using Convolutional Networks
- Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
- Pose Induction for Novel Object Categories
- Deep Kinematic Pose Regression
- Click Here: Human-Localized Keypoints as Guidance for Viewpoint Estimation
- Convolutional Networks for Object Category and 3D Pose Estimation from 2D Images