Semantic keypoint-based pose estimation from single RGB frames

Abstract

This paper presents an approach to estimating the continuous 6-DoF pose of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior investigators, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training-image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Additionally, we accompany our main pipeline with a technique for semi-automatic data generation from unlabeled videos. This procedure allows us to train the learnable components of our method with minimal manual intervention in the labeling process. Empirically, we show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios even against a cluttered background. We apply our approach both to several, existing, large-scale datasets - including PASCAL3D+, LineMOD-Occluded, YCB-Video, and TUD-Light - and, using our labeling pipeline, to a new dataset with novel object classes that we introduce here. Extensive empirical evaluations show that our approach is able to provide pose estimation results comparable to the state of the art.

arXiv commenthttps://sites.google.com/view/rcta-object-keypoints-dataset/home. arXiv admin note: substantial text overlap with arXiv:1703.04670

References in corpus (25)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• DeepPose: Human Pose Estimation via Deep Neural Networks
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Unsupervised Data Augmentation for Consistency Training
• Deeply-Supervised Nets
• Manopt, a Matlab toolbox for optimization on manifolds
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• Do Convnets Learn Correspondence?
• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
• Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image
• kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation
• PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation
• Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
• BOP: Benchmark for 6D Object Pose Estimation
• Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge
• MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
• HybridPose: 6D Object Pose Estimation under Hybrid Representations
• Convolutional Neural Networks for joint object detection and pose estimation: A comparative study
• 6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints
• Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
• Reconstructing Vechicles from a Single Image: Shape Priors for Road Scene Understanding
• KETO: Learning Keypoint Representations for Tool Manipulation
• Self6D: Self-Supervised Monocular 6D Object Pose Estimation
• GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision
• CRAVES: Controlling Robotic Arm with a Vision-based Economic System

Cited by in corpus (1)

• Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints