ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields

Abstract

High-resolution optical satellite sensors, combined with dense stereo algorithms, have made it possible to reconstruct 3D city models from space. However, these models are, in practice, rather noisy and tend to miss small geometric features that are clearly visible in the images. We argue that one reason for the limited quality may be a too early, heuristic reduction of the triangulated 3D point cloud to an explicit height field or surface mesh. To make full use of the point cloud and the underlying images, we introduce ImpliCity, a neural representation of the 3D scene as an implicit, continuous occupancy field, driven by learned embeddings of the point cloud and a stereo pair of ortho-photos. We show that this representation enables the extraction of high-quality DSMs: with image resolution 0.5m, ImpliCity reaches a median height error of $\approx$0.7m and outperforms competing methods, especially w.r.t. building reconstruction, featuring intricate roof details, smooth surfaces, and straight, regular outlines.

arXiv commentAccepted for publication in the International Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences (camera-ready version including keywords + supplementary material)

References in corpus (8)

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
• Implicit Neural Representations with Periodic Activation Functions
• PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
• Learning Implicit Fields for Generative Shape Modeling
• ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images
• Machine-learned 3D Building Vectorization from Satellite Imagery
• S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling