IAN: Iterated Adaptive Neighborhoods for manifold learning and dimensionality estimation

Abstract

Invoking the manifold assumption in machine learning requires knowledge of the manifold's geometry and dimension, and theory dictates how many samples are required. However, in applications data are limited, sampling may not be uniform, and manifold properties are unknown and (possibly) non-pure; this implies that neighborhoods must adapt to the local structure. We introduce an algorithm for inferring adaptive neighborhoods for data given by a similarity kernel. Starting with a locally-conservative neighborhood (Gabriel) graph, we sparsify it iteratively according to a weighted counterpart. In each step, a linear program yields minimal neighborhoods globally and a volumetric statistic reveals neighbor outliers likely to violate manifold geometry. We apply our adaptive neighborhoods to non-linear dimensionality reduction, geodesic computation and dimension estimation. A comparison against standard algorithms using, e.g., k-nearest neighbors, demonstrates their usefulness. Code for our algorithm will be available at https://github.com/dyballa/IAN

arXiv comment69 pages, 35 figures

References in corpus (7)

• Scikit-learn: Machine Learning in Python
• Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding
• Estimating the intrinsic dimension of datasets by a minimal neighborhood information
• Visualizing Large-scale and High-dimensional Data
• Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization
• Nonparametric forecasting of low-dimensional dynamical systems
• Learning mixtures of separated nonspherical Gaussians