Manifold Topology Divergence: a Framework for Comparing Data Manifolds
arXiv:2106.04024
Abstract
We develop a framework for comparing data manifolds, aimed, in particular, towards the evaluation of deep generative models. We describe a novel tool, Cross-Barcode(P,Q), that, given a pair of distributions in a high-dimensional space, tracks multiscale topology spacial discrepancies between manifolds on which the distributions are concentrated. Based on the Cross-Barcode, we introduce the Manifold Topology Divergence score (MTop-Divergence) and apply it to assess the performance of deep generative models in various domains: images, 3D-shapes, time-series, and on different datasets: MNIST, Fashion MNIST, SVHN, CIFAR10, FFHQ, chest X-ray images, market stock data, ShapeNet. We demonstrate that the MTop-Divergence accurately detects various degrees of mode-dropping, intra-mode collapse, mode invention, and image disturbance. Our algorithm scales well (essentially linearly) with the increase of the dimension of the ambient high-dimensional space. It is one of the first TDA-based practical methodologies that can be applied universally to datasets of different sizes and dimensions, including the ones on which the most recent GANs in the visual domain are trained. The proposed method is domain agnostic and does not rely on pre-trained networks.
References in corpus (12)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Geometric deep learning: going beyond Euclidean data
- Conditional Image Synthesis With Auxiliary Classifier GANs
- CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection
- Learning Representations and Generative Models for 3D Point Clouds
- A note on the evaluation of generative models
- Demystifying MMD GANs
- Classification Accuracy Score for Conditional Generative Models
- Geometry Score: A Method For Comparing Generative Adversarial Networks
- A domain agnostic measure for monitoring and evaluating GANs
- The Shape of Data: Intrinsic Distance for Data Distributions
- Skill Rating for Generative Models