Training Invertible Neural Networks as Autoencoders
arXiv:2303.11239 · doi:10.1007/978-3-030-33676-9_31
Abstract
Autoencoders are able to learn useful data representations in an unsupervised matter and have been widely used in various machine learning and computer vision tasks. In this work, we present methods to train Invertible Neural Networks (INNs) as (variational) autoencoders which we call INN (variational) autoencoders. Our experiments on MNIST, CIFAR and CelebA show that for low bottleneck sizes our INN autoencoder achieves results similar to the classical autoencoder. However, for large bottleneck sizes our INN autoencoder outperforms its classical counterpart. Based on the empirical results, we hypothesize that INN autoencoders might not have any intrinsic information loss and thereby are not bounded to a maximal number of layers (depth) after which only suboptimal results can be achieved.
arXiv commentConference Paper at GCPR2019
References in corpus (22)
- Adam: A Method for Stochastic Optimization
- Conditional Generative Adversarial Nets
- Fully Convolutional Networks for Semantic Segmentation
- Tutorial on Variational Autoencoders
- NICE: Non-linear Independent Components Estimation
- Opening the Black Box of Deep Neural Networks via Information
- Density estimation using Real NVP
- Adversarially Learned Inference
- Deep Variational Information Bottleneck
- Fast R-CNN
- The Reversible Residual Network: Backpropagation Without Storing Activations
- Glow: Generative Flow with Invertible 1x1 Convolutions
- Invertible Residual Networks
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Excessive Invariance Causes Adversarial Vulnerability
- i-RevNet: Deep Invertible Networks
- Comparison of Maximum Likelihood and GAN-based training of Real NVPs
- Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
- Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration
- Training Generative Reversible Networks
- Analysis of Invariance and Robustness via Invertibility of ReLU-Networks
- Invertible Autoencoder for domain adaptation