Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation

Abstract

Learning generative probabilistic models is a core problem in machine learning, which presents significant challenges due to the curse of dimensionality. This paper proposes a joint dimensionality reduction and non-parametric density estimation framework, using a novel estimator that can explicitly capture the underlying distribution of appropriate reduced-dimension representations of the input data. The idea is to jointly design a nonlinear dimensionality reducing auto-encoder to model the training data in terms of a parsimonious set of latent random variables, and learn a canonical low-rank tensor model of the joint distribution of the latent variables in the Fourier domain. The proposed latent density model is non-parametric and universal, as opposed to the predefined prior that is assumed in variational auto-encoders. Joint optimization of the auto-encoder and the latent density estimator is pursued via a formulation which learns both by minimizing a combination of the negative log-likelihood in the latent domain and the auto-encoder reconstruction loss. We demonstrate that the proposed model achieves very promising results on toy, tabular, and image datasets on regression tasks, sampling, and anomaly detection.

References in corpus (7)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• WaveNet: A Generative Model for Raw Audio
• NICE: Non-linear Independent Components Estimation
• MADE: Masked Autoencoder for Distribution Estimation
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• Diagnosing and Enhancing VAE Models
• Gaussianization Flows

Cited by in corpus (1)

• Low-rank Characteristic Tensor Density Estimation Part I: Foundations