3DRM:Pair-wise relation module for 3D object detection

Abstract

Context has proven to be one of the most important factors in object layout reasoning for 3D scene understanding. Existing deep contextual models either learn holistic features for context encoding or rely on pre-defined scene templates for context modeling. We argue that scene understanding benefits from object relation reasoning, which is capable of mitigating the ambiguity of 3D object detections and thus helps locate and classify the 3D objects more accurately and robustly. To achieve this, we propose a novel 3D relation module (3DRM) which reasons about object relations at pair-wise levels. The 3DRM predicts the semantic and spatial relationships between objects and extracts the object-wise relation features. We demonstrate the effects of 3DRM by plugging it into proposal-based and voting-based 3D object detection pipelines, respectively. Extensive evaluations show the effectiveness and generalization of 3DRM on 3D object detection. Our source code is available at https://github.com/lanlan96/3DRM.

arXiv comment13 pages, 8 figures

References in corpus (17)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• A simple neural network module for relational reasoning
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
• Relation-Shape Convolutional Neural Network for Point Cloud Analysis
• MLCVNet: Multi-Level Context VoteNet for 3D Object Detection
• MUREL: Multimodal Relational Reasoning for Visual Question Answering
• Spatial Pyramid Based Graph Reasoning for Semantic Segmentation
• 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
• A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes
• Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image
• Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery
• Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction