PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds

Abstract

We propose a novel end-to-end deep scene flow model, called PointPWC-Net, on 3D point clouds in a coarse-to-fine fashion. Flow computed at the coarse level is upsampled and warped to a finer level, enabling the algorithm to accommodate for large motion without a prohibitive search space. We introduce novel cost volume, upsampling, and warping layers to efficiently handle 3D point cloud data. Unlike traditional cost volumes that require exhaustively computing all the cost values on a high-dimensional grid, our point-based formulation discretizes the cost volume onto input 3D points, and a PointConv operation efficiently computes convolutions on the cost volume. Experiment results on FlyingThings3D outperform the state-of-the-art by a large margin. We further explore novel self-supervised losses to train our model and achieve comparable results to state-of-the-art trained with supervised loss. Without any fine-tuning, our method also shows great generalization ability on KITTI Scene Flow 2015 dataset, outperforming all previous methods.

arXiv commentECCV 2020

References in corpus (2)

• Point-Set Registration: Coherent Point Drift
• 3D non-rigid registration using color: Color Coherent Point Drift

Cited by in corpus (12)

• Hierarchical Attention Learning of Scene Flow in 3D Point Clouds
• Onboard dynamic-object detection and tracking for autonomous robot navigation with RGB-D camera
• FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimation
• PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization
• MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
• Attention Models for Point Clouds in Deep Learning: A Survey
• CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation
• Temporal LiDAR Frame Prediction for Autonomous Driving
• FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds
• DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction
• Dual Geometric Graph Network (DG2N) -- Iterative network for deformable shape alignment
• Do not trust the neighbors! Adversarial Metric Learning for Self-Supervised Scene Flow Estimation