Autoregressive Models: What Are They Good For?
arXiv:1910.07737
Abstract
Autoregressive (AR) models have become a popular tool for unsupervised learning, achieving state-of-the-art log likelihood estimates. We investigate the use of AR models as density estimators in two settings -- as a learning signal for image translation, and as an outlier detector -- and find that these density estimates are much less reliable than previously thought. We examine the underlying optimization issues from both an empirical and theoretical perspective, and provide a toy example that illustrates the problem. Overwhelmingly, we find that density estimates do not correlate with perceptual quality and are unhelpful for downstream tasks.
arXiv commentAccepted for the Information Theory and Machine Learning workshop at NeurIPS 2019
References in corpus (5)
- Pixel Recurrent Neural Networks
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Density estimation using Real NVP
- A note on the evaluation of generative models
- Merger-driven Fueling of Active Galactic Nuclei: Six Dual and Offset Active Galactic Nuclei Discovered with Chandra and Hubble Space Telescope Observations