On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding

Abstract

We develop theory for nonlinear dimensionality reduction (NLDR). A number of NLDR methods have been developed, but there is limited understanding of how these methods work and the relationships between them. There is limited basis for using existing NLDR theory for deriving new algorithms. We provide a novel framework for analysis of NLDR via a connection to the statistical theory of linear smoothers. This allows us to both understand existing methods and derive new ones. We use this connection to smoothing to show that asymptotically, existing NLDR methods correspond to discrete approximations of the solutions of sets of differential equations given a boundary condition. In particular, we can characterize many existing methods in terms of just three limiting differential operators and boundary conditions. Our theory also provides a way to assert that one method is preferable to another; indeed, we show Local Tangent Space Alignment is superior within a class of methods that assume a global coordinate chart defines an isometric embedding of the manifold.

References in corpus (1)

• Consistency of spectral clustering

Cited by in corpus (4)

• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Manifold Learning via Manifold Deflation
• A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks
• Differential Similarity in Higher Dimensional Spaces: Theory and Applications