Recurrent Estimation of Distributions
arXiv:1705.10750
Abstract
This paper presents the recurrent estimation of distributions (RED) for modeling real-valued data in a semiparametric fashion. RED models make two novel uses of recurrent neural networks (RNNs) for density estimation of general real-valued data. First, RNNs are used to transform input covariates into a latent space to better capture conditional dependencies in inputs. After, an RNN is used to compute the conditional distributions of the latent covariates. The resulting model is efficient to train, compute, and sample from, whilst producing normalized pdfs. The effectiveness of RED is shown via several real-world data experiments. Our results show that RED models achieve a lower held-out negative log-likelihood than other neural network approaches across multiple dataset sizes and dimensionalities. Further context of the efficacy of RED is provided by considering anomaly detection tasks, where we also observe better performance over alternative models.
References in corpus (7)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- NICE: Non-linear Independent Components Estimation
- Density estimation using Real NVP
- Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
- A Deep and Tractable Density Estimator
- Forest Density Estimation