Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges

Abstract

An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of 3D scene understanding is the availability of large-scale and richly annotated datasets. However, publicly available datasets are either in relative small spatial scales or have limited semantic annotations due to the expensive cost of data acquisition and data annotation, which severely limits the development of fine-grained semantic understanding in the context of 3D point clouds. In this paper, we present an urban-scale photogrammetric point cloud dataset with nearly three billion richly annotated points, which is three times the number of labeled points than the existing largest photogrammetric point cloud dataset. Our dataset consists of large areas from three UK cities, covering about 7.6 km^2 of the city landscape. In the dataset, each 3D point is labeled as one of 13 semantic classes. We extensively evaluate the performance of state-of-the-art algorithms on our dataset and provide a comprehensive analysis of the results. In particular, we identify several key challenges towards urban-scale point cloud understanding. The dataset is available at https://github.com/QingyongHu/SensatUrban.

arXiv commentCVPR 2021, Code: https://github.com/QingyongHu/SensatUrban

References in corpus (13)

• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways
• A2D2: Audi Autonomous Driving Dataset
• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
• Scalability in Perception for Autonomous Driving: Waymo Open Dataset
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
• SceneNet: Understanding Real World Indoor Scenes With Synthetic Data
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding
• SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth
• The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks
• SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances

Cited by in corpus (4)

• Unsupervised Point Cloud Pre-Training via Occlusion Completion
• Efficient Urban-scale Point Clouds Segmentation with BEV Projection
• Semantic Segmentation on Swiss3DCities: A Benchmark Study on Aerial Photogrammetric 3D Pointcloud Dataset
• OpenGF: An Ultra-Large-Scale Ground Filtering Dataset Built Upon Open ALS Point Clouds Around the World