Category-Level Metric Scale Object Shape and Pose Estimation

Abstract

Advances in deep learning recognition have led to accurate object detection with 2D images. However, these 2D perception methods are insufficient for complete 3D world information. Concurrently, advanced 3D shape estimation approaches focus on the shape itself, without considering metric scale. These methods cannot determine the accurate location and orientation of objects. To tackle this problem, we propose a framework that jointly estimates a metric scale shape and pose from a single RGB image. Our framework has two branches: the Metric Scale Object Shape branch (MSOS) and the Normalized Object Coordinate Space branch (NOCS). The MSOS branch estimates the metric scale shape observed in the camera coordinates. The NOCS branch predicts the normalized object coordinate space (NOCS) map and performs similarity transformation with the rendered depth map from a predicted metric scale mesh to obtain 6d pose and size. Additionally, we introduce the Normalized Object Center Estimation (NOCE) to estimate the geometrically aligned distance from the camera to the object center. We validated our method on both synthetic and real-world datasets to evaluate category-level object pose and shape.

arXiv commentIEEE Robotics and Automation Letters (RA-L). Preprint Version. Accepted August, 2021

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer

Cited by in corpus (5)

• Challenges for Monocular 6D Object Pose Estimation in Robotics
• ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation
• DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field
• Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction
• Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference