Autoencoder Image Interpolation by Shaping the Latent Space

Abstract

Autoencoders represent an effective approach for computing the underlying factors characterizing datasets of different types. The latent representation of autoencoders have been studied in the context of enabling interpolation between data points by decoding convex combinations of latent vectors. This interpolation, however, often leads to artifacts or produces unrealistic results during reconstruction. We argue that these incongruities are due to the structure of the latent space and because such naively interpolated latent vectors deviate from the data manifold. In this paper, we propose a regularization technique that shapes the latent representation to follow a manifold that is consistent with the training images and that drives the manifold to be smooth and locally convex. This regularization not only enables faithful interpolation between data points, as we show herein, but can also be used as a general regularization technique to avoid overfitting or to produce new samples for data augmentation.

arXiv commentSubmitted Sept 2020

References in corpus (10)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• mixup: Beyond Empirical Risk Minimization
• Tutorial on Variational Autoencoders
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
• Semantic Facial Expression Editing using Autoencoded Flow
• Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions
• Avoiding Latent Variable Collapse With Generative Skip Models
• On Adversarial Mixup Resynthesis