Disentangled Recurrent Wasserstein Autoencoder

Abstract

Learning disentangled representations leads to interpretable models and facilitates data generation with style transfer, which has been extensively studied on static data such as images in an unsupervised learning framework. However, only a few works have explored unsupervised disentangled sequential representation learning due to challenges of generating sequential data. In this paper, we propose recurrent Wasserstein Autoencoder (R-WAE), a new framework for generative modeling of sequential data. R-WAE disentangles the representation of an input sequence into static and dynamic factors (i.e., time-invariant and time-varying parts). Our theoretical analysis shows that, R-WAE minimizes an upper bound of a penalized form of the Wasserstein distance between model distribution and sequential data distribution, and simultaneously maximizes the mutual information between input data and different disentangled latent factors, respectively. This is superior to (recurrent) VAE which does not explicitly enforce mutual information maximization between input data and disentangled latent representations. When the number of actions in sequential data is available as weak supervision information, R-WAE is extended to learn a categorical latent representation of actions to improve its disentanglement. Experiments on a variety of datasets show that our models outperform other baselines with the same settings in terms of disentanglement and unconditional video generation both quantitatively and qualitatively.

ICLR 2021

References in corpus (12)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Categorical Reparameterization with Gumbel-Softmax
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• A Recurrent Latent Variable Model for Sequential Data
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• On the regularization of Wasserstein GANs
• Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
• From optimal transport to generative modeling: the VEGAN cookbook
• On the Latent Space of Wasserstein Auto-Encoders

Cited by in corpus (4)

• Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAE
• S3RP: Self-Supervised Super-Resolution and Prediction for Advection-Diffusion Process
• Two Stochastic Control Problems In Capital Structure and Portfolio Choice
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System