From Principal Subspaces to Principal Components with Linear Autoencoders

Abstract

The autoencoder is an effective unsupervised learning model which is widely used in deep learning. It is well known that an autoencoder with a single fully-connected hidden layer, a linear activation function and a squared error cost function trains weights that span the same subspace as the one spanned by the principal component loading vectors, but that they are not identical to the loading vectors. In this paper, we show how to recover the loading vectors from the autoencoder weights.

Cited by in corpus (12)

• Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number
• Predicting waves in fluids with deep neural network
• Loss Landscapes of Regularized Linear Autoencoders
• Autoencoders for dimensionality reduction in molecular dynamics: collective variable dimension, biasing and transition states
• An astronomical image content-based recommendation system using combined deep learning models in a fully unsupervised mode
• Dimension Estimation Using Autoencoders
• Why Unsupervised Deep Networks Generalize
• Supervised Dimensionality Reduction for Big Data
• The Power of Contrast for Feature Learning: A Theoretical Analysis
• Autoencoding Improves Pre-trained Word Embeddings
• Compressed Compressor
• A Mechanism for Producing Aligned Latent Spaces with Autoencoders