Invertible Manifold Learning for Dimension Reduction
arXiv:2010.04012
Abstract
Dimension reduction (DR) aims to learn low-dimensional representations of high-dimensional data with the preservation of essential information. In the context of manifold learning, we define that the representation after information-lossless DR preserves the topological and geometric properties of data manifolds formally, and propose a novel two-stage DR method, called invertible manifold learning (inv-ML) to bridge the gap between theoretical information-lossless and practical DR. The first stage includes a homeomorphic sparse coordinate transformation to learn low-dimensional representations without destroying topology and a local isometry constraint to preserve local geometry. In the second stage, a linear compression is implemented for the trade-off between the target dimension and the incurred information loss in excessive DR scenarios. Experiments are conducted on seven datasets with a neural network implementation of inv-ML, called i-ML-Enc. Empirically, i-ML-Enc achieves invertible DR in comparison with typical existing methods as well as reveals the characteristics of the learned manifolds. Through latent space interpolation on real-world datasets, we find that the reliability of tangent space approximated by the local neighborhood is the key to the success of manifold-based DR algorithms.
arXiv commentECML-PKDD 2021 camera-ready. 15 pages (main) with 10 pages appendix
References in corpus (7)
- Scikit-learn: Machine Learning in Python
- Auto-Encoding Variational Bayes
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- NICE: Non-linear Independent Components Estimation
- Extendable and invertible manifold learning with geometry regularized autoencoders
- Training Invertible Neural Networks as Autoencoders
- Markov-Lipschitz Deep Learning