Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder
arXiv:1906.00536 · doi:10.3390/e24030423
Abstract
We present a coupled Variational Auto-Encoder (VAE) method that improves the accuracy and robustness of the probabilistic inferences on represented data. The new method models the dependency between input feature vectors (images) and weighs the outliers with a higher penalty by generalizing the original loss function to the coupled entropy function, using the principles of nonlinear statistical coupling. We evaluate the performance of the coupled VAE model using the MNIST dataset. Compared with the traditional VAE algorithm, the output images generated by the coupled VAE method are clearer and less blurry. The visualization of the input images embedded in 2D latent variable space provides a deeper insight into the structure of new model with coupled loss function: the latent variable has a smaller deviation and a more compact latent space generates the output values. We analyze the histogram of the likelihoods of the input images using the generalized mean, which measures the model's accuracy as a function of the relative risk. The neutral accuracy, which is the geometric mean and is consistent with a measure of the Shannon cross-entropy, is improved. The robust accuracy, measured by the -2/3 generalized mean, is also improved.
arXiv comment19 pages, 11 figures
References in corpus (22)
- Auto-Encoding Variational Bayes
- Variational Inference: A Review for Statisticians
- Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications
- Adversarial Feature Learning
- Adversarially Learned Inference
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- NVAE: A Deep Hierarchical Variational Autoencoder
- Learning with a Wasserstein Loss
- Stability of Tsallis antropy and instabilities of Renyi and normalized Tsallis entropies: A basis for q-exponential distributions
- Autoencoding Variational Inference For Topic Models
- Deep Probabilistic Programming
- Stabilizing GAN Training with Multiple Random Projections
- The three faces of entropy for complex systems -- information, thermodynamics and the maxent principle
- Variational Approach and Deformed Derivatives
- Nonlinear statistical coupling
- Variational autoencoder reconstruction of complex many-body physics
- On the average uncertainty for systems with nonlinear coupling
- Robust Variational Autoencoder
- Generalized Statistics Variational Perturbation Approximation using q-Deformed Calculus
- A risk profile for information fusion algorithms
- Reduced Perplexity: A simplified perspective on assessing probabilistic forecasts
- Use of Student's t-Distribution for the Latent Layer in a Coupled Variational Autoencoder