Quantifying point cloud realism through adversarially learned latent representations

Abstract

Judging the quality of samples synthesized by generative models can be tedious and time consuming, especially for complex data structures, such as point clouds. This paper presents a novel approach to quantify the realism of local regions in LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. Inspired by fair networks, we use an adversarial technique to discourage the encoding of dataset-specific information. The resulting metric can assign a quality score to samples without requiring any task specific annotations. In a series of experiments, we confirm the soundness of our metric by applying it in controllable task setups and on unseen data. Additional experiments show reliable interpolation capabilities of the metric between data with varying degree of realism. As one important application, we demonstrate how the local realism score can be used for anomaly detection in point clouds.

arXiv comment2021 German Conference on Pattern Recognition (GCPR). Project Page: http://ltriess.github.io/lidar-metric

References in corpus (13)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Learning Representations and Generative Models for 3D Point Clouds
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• Disentangling factors of variation in deep representations using adversarial training
• An empirical study on evaluation metrics of generative adversarial networks
• LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
• AdaGAN: Boosting Generative Models
• On GANs and GMMs
• Activation Maximization Generative Adversarial Nets
• LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving
• CNN-based synthesis of realistic high-resolution LiDAR data
• Decoupled Learning for Conditional Adversarial Networks