Contrastive Variational Autoencoder Enhances Salient Features

Abstract

Variational autoencoders are powerful algorithms for identifying dominant latent structure in a single dataset. In many applications, however, we are interested in modeling latent structure and variation that are enriched in a target dataset compared to some background---e.g. enriched in patients compared to the general population. Contrastive learning is a principled framework to capture such enriched variation between the target and background, but state-of-the-art contrastive methods are limited to linear models. In this paper, we introduce the contrastive variational autoencoder (cVAE), which combines the benefits of contrastive learning with the power of deep generative models. The cVAE is designed to identify and enhance salient latent features. The cVAE is trained on two related but unpaired datasets, one of which has minimal contribution from the salient latent features. The cVAE explicitly models latent features that are shared between the datasets, as well as those that are enriched in one dataset relative to the other, which allows the algorithm to isolate and enhance the salient latent features. The algorithm is straightforward to implement, has a similar run-time to the standard VAE, and is robust to noise and dataset purity. We conduct experiments across diverse types of data, including gene expression and facial images, showing that the cVAE effectively uncovers latent structure that is salient in a particular analysis.

Submitted to ICML 2019

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Tutorial on Variational Autoencoders
• Disentangling by Factorising
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Domain Separation Networks
• Isolating Sources of Disentanglement in Variational Autoencoders
• Image-to-image translation for cross-domain disentanglement

Cited by in corpus (8)

• Supporting Analysis of Dimensionality Reduction Results with Contrastive Learning
• UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders
• Autism Spectrum Disorder Classification with Interpretability in Children based on Structural MRI Features Extracted using Contrastive Variational Autoencoder
• Uncertainty Inspired RGB-D Saliency Detection
• Capturing patterns of variation unique to a specific dataset
• A Visual Analytics Framework for Contrastive Network Analysis
• SepVAE: a contrastive VAE to separate pathological patterns from healthy ones
• Noise Contrastive Variational Autoencoders