Task-Aware Sampling Layer for Point-Wise Analysis

Abstract

Sampling, grouping, and aggregation are three important components in the multi-scale analysis of point clouds. In this paper, we present a novel data-driven sampler learning strategy for point-wise analysis tasks. Unlike the widely used sampling technique, Farthest Point Sampling (FPS), we propose to learn sampling and downstream applications jointly. Our key insight is that uniform sampling methods like FPS are not always optimal for different tasks: sampling more points around boundary areas can make the point-wise classification easier for segmentation. Towards this end, we propose a novel sampler learning strategy that learns sampling point displacement supervised by task-related ground truth information and can be trained jointly with the underlying tasks. We further demonstrate our methods in various point-wise analysis tasks, including semantic part segmentation, point cloud completion, and keypoint detection. Our experiments show that jointly learning of the sampler and task brings better performance than using FPS in various point-based networks.

arXiv comment14 pages, 13 figures and 14 tables

References in corpus (7)

• Submanifold Sparse Convolutional Networks
• P2P-NET: Bidirectional Point Displacement Net for Shape Transform
• RGCNN: Regularized Graph CNN for Point Cloud Segmentation
• A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
• Learning Part Boundaries from 3D Point Clouds
• Skeleton-bridged Point Completion: From Global Inference to Local Adjustment
• ME-PCN: Point Completion Conditioned on Mask Emptiness