Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment

Abstract

Nonlinear manifold learning from unorganized data points is a very challenging unsupervised learning and data visualization problem with a great variety of applications. In this paper we present a new algorithm for manifold learning and nonlinear dimension reduction. Based on a set of unorganized data points sampled with noise from the manifold, we represent the local geometry of the manifold using tangent spaces learned by fitting an affine subspace in a neighborhood of each data point. Those tangent spaces are aligned to give the internal global coordinates of the data points with respect to the underlying manifold by way of a partial eigendecomposition of the neighborhood connection matrix. We present a careful error analysis of our algorithm and show that the reconstruction errors are of second-order accuracy. We illustrate our algorithm using curves and surfaces both in 2D/3D and higher dimensional Euclidean spaces, and 64-by-64 pixel face images with various pose and lighting conditions. We also address several theoretical and algorithmic issues for further research and improvements.

Cited by in corpus (32)

• Efficient Estimation of Mutual Information for Strongly Dependent Variables
• Manifold Partition Discriminant Analysis
• Patch alignment manifold matting
• Tangent Bundle Manifold Learning via Grassmann&Stiefel Eigenmaps
• Tangent-based manifold approximation with locally linear models
• Multiscale Dictionary Learning: Non-Asymptotic Bounds and Robustness
• Universal local parametrizations via heat kernels and eigenfunctions of the Laplacian
• PCA-Based Out-of-Sample Extension for Dimensionality Reduction
• Low Rank Representation on Grassmann Manifolds: An Extrinsic Perspective
• Chart Auto-Encoders for Manifold Structured Data
• SVM and Dimensionality Reduction in Cognitive Radio with Experimental Validation
• Kernelized Low Rank Representation on Grassmann Manifolds
• Kernelized LRR on Grassmann Manifolds for Subspace Clustering
• A study of the classification of low-dimensional data with supervised manifold learning
• Clustering, Coding, and the Concept of Similarity
• A New Method for Performance Analysis in Nonlinear Dimensionality Reduction
• Multi-View Kernel Consensus For Data Analysis
• Parameterization of non-linear manifolds
• Multiscale regression on unknown manifolds
• Ensemble Manifold Segmentation for Model Distillation and Semi-supervised Learning
• Applying Ricci Flow to High Dimensional Manifold Learning
• A Tangent Distance Preserving Dimensionality Reduction Algorithm
• Curvature-aware Manifold Learning
• Tangent space estimation for smooth embeddings of Riemannian manifolds
• Multiview Hessian regularized logistic regression for action recognition
• Manifold Fitting in Ambient Space
• Mathematical Analysis on Out-of-Sample Extensions
• Localized LRR on Grassmann Manifolds: An Extrinsic View
• Learning Low-dimensional Manifolds for Scoring of Tissue Microarray Images
• Quadratic Projection Based Feature Extraction with Its Application to Biometric Recognition
• Differential Similarity in Higher Dimensional Spaces: Theory and Applications
• Partial Sum Minimization of Singular Values Representation on Grassmann Manifolds